Skip to content

Commit a3c8e86

Browse files
hjmjohnsonhjmjohnson
committed
ENH: Refactor and enhance the CI testing infrastructure
1) Improve travis build script for use outside travis. Allow the script used for CI builds to also be used locally in a similar manner to the CI use of the scrips 2) Add ctest compatible testing and CDASH support Report testing and building results to https://my.cdash.org/index.php?project=jsoncpp NOTE: The new ctest infrastructure is not yet robust on winodws Do no yet enable the new features for running test with ctest on windows platform. The previous behaviors are maintainted, but enhance test reporting from windows is not yet supported. 3) Add a cmake coverage testing option Ensure that cmake builds on linux are tested. Ensure that code coverage is reported. 4) Move conditional environment checking into the matrix Avoid multiple places where conditional logic is used to change compiler behavior. As more test environments are created fromt the travis.yml matrix, all settings should be obvious from that one location. 5) Tests with known regressions from the jsonchecker are suppressed Tests that are known to pass with jsoncpp more lenient syntax enforcement are exluded from tests in test/runjsontests.py
1 parent 10a1a38 commit a3c8e86

12 files changed

Lines changed: 341 additions & 55 deletions

File tree

‎.travis.yml‎

Lines changed: 42 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -26,19 +26,52 @@ matrix:
2626
allow_failures:
2727
- os: osx
2828
include:
29-
- os: osx
29+
- name: Mac clang meson static release testing
30+
os: osx
3031
osx_image: xcode9.4
3132
compiler: clang
32-
env: LIB_TYPE=static BUILD_TYPE=release
33-
- os: linux
33+
env:
34+
CXX="clang++-3.5"
35+
CC="clang-3.5"
36+
LIB_TYPE=static
37+
BUILD_TYPE=release
38+
script: ./.travis_scripts/meson_builder.sh
39+
- name: trusty clang meson static release testing
40+
os: linux
3441
dist: trusty
3542
compiler: clang
36-
env: LIB_TYPE=static BUILD_TYPE=release
43+
env:
44+
CXX="clang++-3.5"
45+
CC="clang-3.5"
46+
LIB_TYPE=static
47+
BUILD_TYPE=release
48+
# before_install and install steps only needed for linux meson builds
49+
before_install:
50+
- source ./.travis_scripts/travis.before_install.${TRAVIS_OS_NAME}.sh
51+
install:
52+
- source ./.travis_scripts/travis.install.${TRAVIS_OS_NAME}.sh
53+
script: ./.travis_scripts/meson_builder.sh
54+
- name: xenial gcc cmake coverage
55+
os: linux
56+
dist: xenial
57+
compiler: gcc
58+
env:
59+
CXX=g++
60+
CC=gcc
61+
DO_Coverage=ON
62+
BUILD_TOOL="Unix Makefiles"
63+
BUILD_TYPE=Debug
64+
LIB_TYPE=shared
65+
DESTDIR=/tmp/cmake_json_cpp
66+
script: ./.travis_scripts/cmake_builder.sh
67+
# Valgrind has too many false positives from the python wrapping. Need a good suppression file
68+
# - name: xenial gcc cmake coverage
69+
# os: linux
70+
# dist: xenial
71+
# compiler: gcc
72+
# env: DO_MemCheck=ON CXX=/usr/bin/g++ BUILD_TOOL="Unix Makefiles" BUILD_TYPE=Debug LIB_TYPE=shared DESTDIR=/tmp/cmake_json_cpp
73+
# script: ./.travis_scripts/cmake_builder.sh
3774
notifications:
3875
email: false
3976

40-
before_install:
41-
- source ./.travis_scripts/travis.before_install.${TRAVIS_OS_NAME}.sh
42-
install:
43-
- source ./.travis_scripts/travis.install.${TRAVIS_OS_NAME}.sh
44-
script: ./.travis_scripts/travis.sh
77+

‎.travis_scripts/cmake_builder.sh‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
#!/usr/bin/env sh
2+
# This script can be used on the command line directly to configure several
3+
# different build environments.
4+
# This is called by `.travis.yml` via Travis CI.
5+
# Travis supplies $TRAVIS_OS_NAME.
6+
# http://docs.travis-ci.com/user/multi-os/
7+
# Our .travis.yml also defines:
8+
9+
# - BUILD_TYPE=Release/Debug
10+
# - LIB_TYPE=static/shared
11+
#
12+
# Optional environmental variables
13+
# - DESTDIR <- used for setting the install prefix
14+
# - BUILD_TOOL=["Unix Makefile"|"Ninja"]
15+
# - BUILDNAME <-- how to identify this build on the dashboard
16+
# - DO_MemCheck <- if set, try to use valgrind
17+
# - DO_Coverage <- if set, try to do dashboard coverage testing
18+
#
19+
20+
env_set=1
21+
if ${BUILD_TYPE+false}; then
22+
echo "BUILD_TYPE not set in environment."
23+
env_set=0
24+
fi
25+
if ${LIB_TYPE+false}; then
26+
echo "LIB_TYPE not set in environment."
27+
env_set=0
28+
fi
29+
if ${CXX+false}; then
30+
echo "CXX not set in environment."
31+
env_set=0
32+
fi
33+
34+
35+
if [ ${env_set} -eq 0 ]; then
36+
echo "USAGE: CXX=$(which clang++) BUILD_TYPE=[Release|Debug] LIB_TYPE=[static|shared] $0"
37+
echo ""
38+
echo "Examples:"
39+
echo " CXX=$(which clang++) BUILD_TYPE=Release LIB_TYPE=shared DESTDIR=/tmp/cmake_json_cpp $0"
40+
echo " CXX=$(which clang++) BUILD_TYPE=Debug LIB_TYPE=shared DESTDIR=/tmp/cmake_json_cpp $0"
41+
echo " CXX=$(which clang++) BUILD_TYPE=Release LIB_TYPE=static DESTDIR=/tmp/cmake_json_cpp $0"
42+
echo " CXX=$(which clang++) BUILD_TYPE=Debug LIB_TYPE=static DESTDIR=/tmp/cmake_json_cpp $0"
43+
44+
echo " CXX=$(which g++) BUILD_TYPE=Release LIB_TYPE=shared DESTDIR=/tmp/cmake_json_cpp $0"
45+
echo " CXX=$(which g++) BUILD_TYPE=Debug LIB_TYPE=shared DESTDIR=/tmp/cmake_json_cpp $0"
46+
echo " CXX=$(which g++) BUILD_TYPE=Release LIB_TYPE=static DESTDIR=/tmp/cmake_json_cpp $0"
47+
echo " CXX=$(which g++) BUILD_TYPE=Debug LIB_TYPE=static DESTDIR=/tmp/cmake_json_cpp $0"
48+
49+
exit -1
50+
fi
51+
52+
if ${DESTDIR+false}; then
53+
DESTDIR="/usr/local"
54+
fi
55+
56+
# -e: fail on error
57+
# -v: show commands
58+
# -x: show expanded commands
59+
set -vex
60+
61+
env | sort
62+
63+
which cmake
64+
cmake --version
65+
66+
echo ${CXX}
67+
${CXX} --version
68+
_COMPILER_NAME=`basename ${CXX}`
69+
if [ "${BUILD_TYPE}" == "shared" ]; then
70+
_CMAKE_BUILD_SHARED_LIBS=ON
71+
else
72+
_CMAKE_BUILD_SHARED_LIBS=OFF
73+
fi
74+
75+
CTEST_TESTING_OPTION="-D ExperimentalTest"
76+
# - DO_MemCheck <- if set, try to use valgrind
77+
if ! ${DO_MemCheck+false}; then
78+
valgrind --version
79+
CTEST_TESTING_OPTION="-D ExperimentalMemCheck"
80+
else
81+
# - DO_Coverage <- if set, try to do dashboard coverage testing
82+
if ! ${DO_Coverage+false}; then
83+
export CXXFLAGS="-fprofile-arcs -ftest-coverage"
84+
export LDFLAGS="-fprofile-arcs -ftest-coverage"
85+
CTEST_TESTING_OPTION="-D ExperimentalTest -D ExperimentalCoverage"
86+
#gcov --version
87+
fi
88+
fi
89+
90+
# Ninja = Generates build.ninja files.
91+
if ${BUILD_TOOL+false}; then
92+
BUILD_TOOL="Ninja"
93+
export _BUILD_EXE=ninja
94+
which ninja
95+
ninja --version
96+
else
97+
# Unix Makefiles = Generates standard UNIX makefiles.
98+
export _BUILD_EXE=make
99+
fi
100+
101+
_BUILD_DIR_NAME="build-cmake_${BUILD_TYPE}_${LIB_TYPE}_${_COMPILER_NAME}_${_BUILD_EXE}"
102+
mkdir -p ${_BUILD_DIR_NAME}
103+
cd "${_BUILD_DIR_NAME}"
104+
if ${BUILDNAME+false}; then
105+
_HOSTNAME=`hostname -s`
106+
BUILDNAME="${_HOSTNAME}_${BUILD_TYPE}_${LIB_TYPE}_${_COMPILER_NAME}_${_BUILD_EXE}"
107+
fi
108+
cmake \
109+
-G "${BUILD_TOOL}" \
110+
-DBUILDNAME:STRING="${BUILDNAME}" \
111+
-DCMAKE_CXX_COMPILER:PATH=${CXX} \
112+
-DCMAKE_BUILD_TYPE:STRING=${BUILD_TYPE} \
113+
-DBUILD_SHARED_LIBS:BOOL=${_CMAKE_BUILD_SHARED_LIBS} \
114+
-DCMAKE_INSTALL_PREFIX:PATH=${DESTDIR} \
115+
../
116+
117+
ctest -C ${BUILD_TYPE} -D ExperimentalStart -D ExperimentalConfigure -D ExperimentalBuild ${CTEST_TESTING_OPTION} -D ExperimentalSubmit
118+
# Final step is to verify that installation succeeds
119+
cmake --build . --config ${BUILD_TYPE} --target install
120+
121+
if [ "${DESTDIR}" != "/usr/local" ]; then
122+
${_BUILD_EXE} install
123+
fi
124+
cd -
125+
126+
if ${CLEANUP+false}; then
127+
echo "Skipping cleanup: build directory will persist."
128+
else
129+
rm -r "${_BUILD_DIR_NAME}"
130+
fi

‎.travis_scripts/meson_builder.sh‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
#!/usr/bin/env sh
2+
# This script can be used on the command line directly to configure several
3+
# different build environments.
4+
# This is called by `.travis.yml` via Travis CI.
5+
# Travis supplies $TRAVIS_OS_NAME.
6+
# http://docs.travis-ci.com/user/multi-os/
7+
# Our .travis.yml also defines:
8+
9+
# - BUILD_TYPE=release/debug
10+
# - LIB_TYPE=static/shared
11+
12+
env_set=1
13+
if ${BUILD_TYPE+false}; then
14+
echo "BUILD_TYPE not set in environment."
15+
env_set=0
16+
fi
17+
if ${LIB_TYPE+false}; then
18+
echo "LIB_TYPE not set in environment."
19+
env_set=0
20+
fi
21+
if ${CXX+false}; then
22+
echo "CXX not set in environment."
23+
env_set=0
24+
fi
25+
26+
27+
if [ ${env_set} -eq 0 ]; then
28+
echo "USAGE: CXX=$(which clang++) BUILD_TYPE=[release|debug] LIB_TYPE=[static|shared] $0"
29+
echo ""
30+
echo "Examples:"
31+
echo " CXX=$(which clang++) BUILD_TYPE=release LIB_TYPE=shared DESTDIR=/tmp/meson_json_cpp $0"
32+
echo " CXX=$(which clang++) BUILD_TYPE=debug LIB_TYPE=shared DESTDIR=/tmp/meson_json_cpp $0"
33+
echo " CXX=$(which clang++) BUILD_TYPE=release LIB_TYPE=static DESTDIR=/tmp/meson_json_cpp $0"
34+
echo " CXX=$(which clang++) BUILD_TYPE=debug LIB_TYPE=static DESTDIR=/tmp/meson_json_cpp $0"
35+
36+
echo " CXX=$(which g++) BUILD_TYPE=release LIB_TYPE=shared DESTDIR=/tmp/meson_json_cpp $0"
37+
echo " CXX=$(which g++) BUILD_TYPE=debug LIB_TYPE=shared DESTDIR=/tmp/meson_json_cpp $0"
38+
echo " CXX=$(which g++) BUILD_TYPE=release LIB_TYPE=static DESTDIR=/tmp/meson_json_cpp $0"
39+
echo " CXX=$(which g++) BUILD_TYPE=debug LIB_TYPE=static DESTDIR=/tmp/meson_json_cpp $0"
40+
41+
exit -1
42+
fi
43+
44+
if ${DESTDIR+false}; then
45+
DESTDIR="/usr/local"
46+
fi
47+
48+
# -e: fail on error
49+
# -v: show commands
50+
# -x: show expanded commands
51+
set -vex
52+
53+
54+
env | sort
55+
56+
which python3
57+
which meson
58+
which ninja
59+
echo ${CXX}
60+
${CXX} --version
61+
python3 --version
62+
meson --version
63+
ninja --version
64+
_COMPILER_NAME=`basename ${CXX}`
65+
_BUILD_DIR_NAME="build-${BUILD_TYPE}_${LIB_TYPE}_${_COMPILER_NAME}"
66+
meson --buildtype ${BUILD_TYPE} --default-library ${LIB_TYPE} . "${_BUILD_DIR_NAME}"
67+
ninja -v -j 2 -C "${_BUILD_DIR_NAME}"
68+
#ninja -v -j 2 -C "${_BUILD_DIR_NAME}" test
69+
cd "${_BUILD_DIR_NAME}"
70+
meson test --no-rebuild --print-errorlogs
71+
72+
if [ "${DESTDIR}" != "/usr/local" ]; then
73+
ninja install
74+
fi
75+
cd -
76+
77+
if ${CLEANUP+false}; then
78+
echo "Skipping cleanup: build directory will persist."
79+
else
80+
rm -r "${_BUILD_DIR_NAME}"
81+
fi

‎.travis_scripts/travis.install.linux.sh‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,6 @@ wget https://github.com/ninja-build/ninja/releases/download/v1.7.2/ninja-linux.z
44
unzip -q ninja-linux.zip -d build
55

66
pip3 install meson
7-
# /usr/bin/gcc is 4.6 always, but gcc-X.Y is available.
8-
if [[ $CXX = g++ ]]; then export CXX="g++-4.9" CC="gcc-4.9"; fi
9-
# /usr/bin/clang has a conflict with gcc, so use clang-X.Y.
10-
if [[ $CXX = clang++ ]]; then export CXX="clang++-3.5" CC="clang-3.5"; fi
117
echo ${PATH}
128
ls /usr/local
139
ls /usr/local/bin

‎.travis_scripts/travis.sh‎

Lines changed: 0 additions & 34 deletions
This file was deleted.

‎CMakeLists.txt‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,8 +70,6 @@ set( JSONCPP_VERSION ${JSONCPP_VERSION_MAJOR}.${JSONCPP_VERSION_MINOR}.${JSONCPP
7070
message(STATUS "JsonCpp Version: ${JSONCPP_VERSION_MAJOR}.${JSONCPP_VERSION_MINOR}.${JSONCPP_VERSION_PATCH}")
7171
set( JSONCPP_SOVERSION 19 )
7272

73-
enable_testing()
74-
7573
option(JSONCPP_WITH_TESTS "Compile and (for jsoncpp_check) run JsonCpp test executables" ON)
7674
option(JSONCPP_WITH_POST_BUILD_UNITTEST "Automatically run unit-tests as a post build step" ON)
7775
option(JSONCPP_WITH_WARNING_AS_ERROR "Force compilation to fail if a warning occurs" OFF)
@@ -175,8 +173,14 @@ if(JSONCPP_WITH_CMAKE_PACKAGE)
175173
FILE jsoncppConfig.cmake)
176174
endif()
177175

176+
if(JSONCPP_WITH_TESTS)
177+
enable_testing()
178+
include(CTest)
179+
endif()
180+
178181
# Build the different applications
179182
add_subdirectory( src )
180183

181184
#install the includes
182185
add_subdirectory( include )
186+

‎CTestConfig.cmake‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
## This file should be placed in the root directory of your project.
2+
## Then modify the CMakeLists.txt file in the root directory of your
3+
## project to incorporate the testing dashboard.
4+
##
5+
## # The following are required to submit to the CDash dashboard:
6+
## ENABLE_TESTING()
7+
## INCLUDE(CTest)
8+
9+
set(CTEST_PROJECT_NAME "jsoncpp")
10+
set(CTEST_NIGHTLY_START_TIME "01:23:45 UTC")
11+
12+
set(CTEST_DROP_METHOD "https")
13+
set(CTEST_DROP_SITE "my.cdash.org")
14+
set(CTEST_DROP_LOCATION "/submit.php?project=jsoncpp")
15+
set(CTEST_DROP_SITE_CDASH TRUE)

‎README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,11 @@ format to store user input files.
3131
## Contributing to JsonCpp
3232

3333
### Building and testing with Meson/Ninja
34-
Thanks to David Seifert (@SoapGentoo), we (the maintainers) now use [meson](http://mesonbuild.com/) and [ninja](https://ninja-build.org/) to build for debugging, as well as for continuous integration (see [`travis.sh`](travis.sh) ). Other systems may work, but minor things like version strings might break.
34+
Thanks to David Seifert (@SoapGentoo), we (the maintainers) now use
35+
[meson](http://mesonbuild.com/) and [ninja](https://ninja-build.org/) to build
36+
for debugging, as well as for continuous integration (see
37+
[`./travis_scripts/meson_builder.sh`](./travis_scripts/meson_builder.sh) ). Other systems may work, but minor
38+
things like version strings might break.
3539

3640
First, install both meson (which requires Python3) and ninja.
3741
If you wish to install to a directory other than /usr/local, set an environment variable called DESTDIR with the desired path:

‎appveyor.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,11 @@ environment:
1010
build_script:
1111
- cmake --version
1212
- cd c:\projects\jsoncpp
13-
- cmake -G "%CMAKE_GENERATOR%" -DCMAKE_INSTALL_PREFIX=%CD:\=/%/install -DBUILD_SHARED_LIBS=ON .
13+
- cmake -G "%CMAKE_GENERATOR%" -DCMAKE_INSTALL_PREFIX:PATH=%CD:\=/%/install -DBUILD_SHARED_LIBS:BOOL=ON .
14+
# Use ctest to make a dashboard build ctest -D Experimental(Start|Update|Configure|Build|Test|Coverage|MemCheck|Submit)
15+
#NOTE Testing on window is not yet finished - ctest -C Release -D ExperimentalStart -D ExperimentalConfigure -D ExperimentalBuild -D ExperimentalTest -D ExperimentalSubmit
16+
- ctest -C Release -D ExperimentalStart -D ExperimentalConfigure -D ExperimentalBuild -D ExperimentalSubmit
17+
# Final step is to verify that installation succeeds
1418
- cmake --build . --config Release --target install
1519

1620
deploy:

0 commit comments

Comments
 (0)