[jira] [Comment Edited] (MESOS-6229) Default to using hardened compilation flags

["Aaron Wood (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517039#comment-15517039 ] 

Aaron Wood edited comment on MESOS-6229 at 9/23/16 5:25 PM:
------------------------------------------------------------

Looks like there will need to be some fixes made ahead of time before this patch goes in:

/bin/sh ../../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.1.0\" -DPACKAGE_STRING=\"mesos\ 1.1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"1.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -I. -I../../../3rdparty/libprocess  -DBUILD_DIR=\"/Users//Code/src/mesos/build/3rdparty/libprocess\" -I../../../3rdparty/libprocess/include -isystem ../boost-1.53.0 -I../elfio-3.2 -I../glog-0.3.3/src  -I../http-parser-2.6.2 -I../libev-4.22 -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -I../picojson-1.3.0 -I../../../3rdparty/libprocess/../stout/include  -I/usr/local/opt/subversion/include/subversion-1 -I/usr/local/opt/openssl/include -I/usr/local/opt/libevent/include -I/usr/include/apr-1 -I/usr/include/apr-1.0  -Wall -Werror -Wsign-compare -Wformat-security -Wstack-protector -fno-omit-frame-pointer -fstack-protector-strong -pie -fPIE -D_FORTIFY_SOURCE=2 -O3 -g1 -O0 -Wno-unused-local-typedef -std=c++11 -stdlib=libc++ -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c -o libprocess_la-reap.lo `test -f 'src/reap.cpp' || echo '../../../3rdparty/libprocess/'`src/reap.cpp
../../../3rdparty/libprocess/src/profiler.cpp:35:12: error: unused variable 'PROFILE_FILE' [-Werror,-Wunused-const-variable]
const char PROFILE_FILE[] = "perftools.out";
           ^
In file included from ../../../3rdparty/libprocess/src/profiler.cpp:24:
../../../3rdparty/libprocess/include/process/profiler.hpp:80:8: error: private field 'started' is not used [-Werror,-Wunused-private-field]
  bool started;
       ^
2 errors generated.
make[5]: *** [libprocess_la-profiler.lo] Error 1
make[5]: *** Waiting for unfinished jobs....
mv -f .deps/libprocess_la-logging.Tpo .deps/libprocess_la-logging.Plo
mv -f .deps/libprocess_la-io.Tpo .deps/libprocess_la-io.Plo
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.1.0\" "-DPACKAGE_STRING=\"mesos 1.1.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"1.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -I. -I../../../3rdparty/libprocess -DBUILD_DIR=\"/Users//Code/src/mesos/build/3rdparty/libprocess\" -I../../../3rdparty/libprocess/include -isystem ../boost-1.53.0 -I../elfio-3.2 -I../glog-0.3.3/src -I../http-parser-2.6.2 -I../libev-4.22 -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -I../picojson-1.3.0 -I../../../3rdparty/libprocess/../stout/include -I/usr/local/opt/subversion/include/subversion-1 -I/usr/local/opt/openssl/include -I/usr/local/opt/libevent/include -I/usr/include/apr-1 -I/usr/include/apr-1.0 -Wall -Werror -Wsign-compare -Wformat-security -Wstack-protector -fno-omit-frame-pointer -fstack-protector-strong -D_FORTIFY_SOURCE=2 -O3 -g1 -O0 -Wno-unused-local-typedef -std=c++11 -stdlib=libc++ -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c ../../../3rdparty/libprocess/src/reap.cpp  -fno-common -DPIC -o .libs/libprocess_la-reap.o
In file included from ../../../3rdparty/libprocess/src/process.cpp:108:
../../../3rdparty/libprocess/src/encoder.hpp:278:15: error: comparison of integers of different signs: 'off_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Werror,-Wsign-compare]
    if (index >= length) {
        ~~~~~ ^  ~~~~~~
../../../3rdparty/libprocess/src/process.cpp:3501:23: error: comparison of integers of different signs: 'int' and 'size_type' (aka 'unsigned long') [-Werror,-Wsign-compare]
    for (int i = 2; i < tokens.size(); i++) {
                    ~ ^ ~~~~~~~~~~~~~
mv -f .deps/libprocess_la-http.Tpo .deps/libprocess_la-http.Plo
mv -f .deps/libprocess_la-poll_socket.Tpo .deps/libprocess_la-poll_socket.Plo
mv -f .deps/libprocess_la-reap.Tpo .deps/libprocess_la-reap.Plo
mv -f .deps/libprocess_la-metrics.Tpo .deps/libprocess_la-metrics.Plo
2 errors generated.
make[5]: *** [libprocess_la-process.lo] Error 1
make[4]: *** [all-recursive] Error 1
make[3]: *** [all] Error 2
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1


was (Author: aaron.wood):
Looks like there will need to be some fixes made ahead of time before this patch goes in:

```
/bin/sh ../../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.1.0\" -DPACKAGE_STRING=\"mesos\ 1.1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"1.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -I. -I../../../3rdparty/libprocess  -DBUILD_DIR=\"/Users//Code/src/mesos/build/3rdparty/libprocess\" -I../../../3rdparty/libprocess/include -isystem ../boost-1.53.0 -I../elfio-3.2 -I../glog-0.3.3/src  -I../http-parser-2.6.2 -I../libev-4.22 -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -I../picojson-1.3.0 -I../../../3rdparty/libprocess/../stout/include  -I/usr/local/opt/subversion/include/subversion-1 -I/usr/local/opt/openssl/include -I/usr/local/opt/libevent/include -I/usr/include/apr-1 -I/usr/include/apr-1.0  -Wall -Werror -Wsign-compare -Wformat-security -Wstack-protector -fno-omit-frame-pointer -fstack-protector-strong -pie -fPIE -D_FORTIFY_SOURCE=2 -O3 -g1 -O0 -Wno-unused-local-typedef -std=c++11 -stdlib=libc++ -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c -o libprocess_la-reap.lo `test -f 'src/reap.cpp' || echo '../../../3rdparty/libprocess/'`src/reap.cpp
../../../3rdparty/libprocess/src/profiler.cpp:35:12: error: unused variable 'PROFILE_FILE' [-Werror,-Wunused-const-variable]
const char PROFILE_FILE[] = "perftools.out";
           ^
In file included from ../../../3rdparty/libprocess/src/profiler.cpp:24:
../../../3rdparty/libprocess/include/process/profiler.hpp:80:8: error: private field 'started' is not used [-Werror,-Wunused-private-field]
  bool started;
       ^
2 errors generated.
make[5]: *** [libprocess_la-profiler.lo] Error 1
make[5]: *** Waiting for unfinished jobs....
mv -f .deps/libprocess_la-logging.Tpo .deps/libprocess_la-logging.Plo
mv -f .deps/libprocess_la-io.Tpo .deps/libprocess_la-io.Plo
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.1.0\" "-DPACKAGE_STRING=\"mesos 1.1.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"1.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -I. -I../../../3rdparty/libprocess -DBUILD_DIR=\"/Users//Code/src/mesos/build/3rdparty/libprocess\" -I../../../3rdparty/libprocess/include -isystem ../boost-1.53.0 -I../elfio-3.2 -I../glog-0.3.3/src -I../http-parser-2.6.2 -I../libev-4.22 -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -I../picojson-1.3.0 -I../../../3rdparty/libprocess/../stout/include -I/usr/local/opt/subversion/include/subversion-1 -I/usr/local/opt/openssl/include -I/usr/local/opt/libevent/include -I/usr/include/apr-1 -I/usr/include/apr-1.0 -Wall -Werror -Wsign-compare -Wformat-security -Wstack-protector -fno-omit-frame-pointer -fstack-protector-strong -D_FORTIFY_SOURCE=2 -O3 -g1 -O0 -Wno-unused-local-typedef -std=c++11 -stdlib=libc++ -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c ../../../3rdparty/libprocess/src/reap.cpp  -fno-common -DPIC -o .libs/libprocess_la-reap.o
In file included from ../../../3rdparty/libprocess/src/process.cpp:108:
../../../3rdparty/libprocess/src/encoder.hpp:278:15: error: comparison of integers of different signs: 'off_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Werror,-Wsign-compare]
    if (index >= length) {
        ~~~~~ ^  ~~~~~~
../../../3rdparty/libprocess/src/process.cpp:3501:23: error: comparison of integers of different signs: 'int' and 'size_type' (aka 'unsigned long') [-Werror,-Wsign-compare]
    for (int i = 2; i < tokens.size(); i++) {
                    ~ ^ ~~~~~~~~~~~~~
mv -f .deps/libprocess_la-http.Tpo .deps/libprocess_la-http.Plo
mv -f .deps/libprocess_la-poll_socket.Tpo .deps/libprocess_la-poll_socket.Plo
mv -f .deps/libprocess_la-reap.Tpo .deps/libprocess_la-reap.Plo
mv -f .deps/libprocess_la-metrics.Tpo .deps/libprocess_la-metrics.Plo
2 errors generated.
make[5]: *** [libprocess_la-process.lo] Error 1
make[4]: *** [all-recursive] Error 1
make[3]: *** [all] Error 2
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1
```

> Default to using hardened compilation flags
> -------------------------------------------
>
>                 Key: MESOS-6229
>                 URL: https://issues.apache.org/jira/browse/MESOS-6229
>             Project: Mesos
>          Issue Type: Improvement
>            Reporter: Aaron Wood
>            Assignee: Aaron Wood
>            Priority: Minor
>              Labels: c++, clang, gcc, security
>
> Provide a default set of hardened compilation flags to help protect against overflows and other attacks. Apply to libprocess and stout as well. Current set of flags that were discussed on slack to implement:
> -Wformat­-security
> -Wstack-protector
> -fstack-protector-all
> -pie
> -fPIE 
> -D_FORTIFY_SOURCE=2
> -O2 (possibly -O3 for greater optimizations, up for discussion)
> ­-Wl,-z,relro,-z,now
> -fno-omit-frame-pointer
> -fstack-protector-strong (-fstack-protector-all might be overkill, it could be more effective to use this. Requires gcc >= 4.9)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)