You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Bernd Mathiske <be...@mesosphere.io> on 2015/02/08 17:47:42 UTC

Re: Review Request 30774: Fetcher Cache Working Copy for Ben and Bernd

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 8, 2015, 8:47 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Updated with bug fixes in tests.


Repository: mesos


Description
-------

Patches 1-11 and 13-15


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 10, 2015, 1:26 a.m., Adam B wrote:
> > Please update the description with a descriptive commit message.
> > Also explain any testing you did for HDFS fetching (since it's half-broken without the HADOOP_HOME fix).

Done. The description of the HDFS testing is under 30614.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71770
-----------------------------------------------------------


On Feb. 14, 2015, 2:38 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 14, 2015, 2:38 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Adam B <ad...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71770
-----------------------------------------------------------


Please update the description with a descriptive commit message.
Also explain any testing you did for HDFS fetching (since it's half-broken without the HADOOP_HOME fix).

- Adam B


On Feb. 10, 2015, 1:14 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2015, 1:14 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71792
-----------------------------------------------------------


Bad patch!

Reviews applied: [30609, 30606, 30774]

Failed command: make -j3 distcheck

Error:
 make  dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot'
if test -d "mesos-0.22.0"; then find "mesos-0.22.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "mesos-0.22.0" || { sleep 5 && rm -rf "mesos-0.22.0"; }; else :; fi
test -d "mesos-0.22.0" || mkdir "mesos-0.22.0"
 (cd 3rdparty && make  top_distdir=../mesos-0.22.0 distdir=../mesos-0.22.0/3rdparty \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty'
 (cd libprocess && make  top_distdir=../../mesos-0.22.0 distdir=../../mesos-0.22.0/3rdparty/libprocess \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess'
:
test -d "../../mesos-0.22.0/3rdparty/libprocess" || mkdir "../../mesos-0.22.0/3rdparty/libprocess"
 (cd 3rdparty && make  top_distdir=../../../mesos-0.22.0 distdir=../../../mesos-0.22.0/3rdparty/libprocess/3rdparty \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty'
 (cd stout && make  top_distdir=../../../../mesos-0.22.0 distdir=../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout'
:
test -d "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout" || mkdir "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout"
 (cd include && make  top_distdir=../../../../../mesos-0.22.0 distdir=../../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout/include \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[6]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout/include'
make[6]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout/include'
test -n ":" \
	|| find "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout" -type d ! -perm -755 \
		-exec chmod u+rwx,go+rx {} \; -o \
	  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -444 -exec /bin/bash /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout/install-sh -c -m a+r {} {} \; \
	|| chmod -R a+r "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout"
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty'
 (cd include && make  top_distdir=../../../mesos-0.22.0 distdir=../../../mesos-0.22.0/3rdparty/libprocess/include \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/include'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/include'
test -n ":" \
	|| find "../../mesos-0.22.0/3rdparty/libprocess" -type d ! -perm -755 \
		-exec chmod u+rwx,go+rx {} \; -o \
	  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -444 -exec /bin/bash /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/install-sh -c -m a+r {} {} \; \
	|| chmod -R a+r "../../mesos-0.22.0/3rdparty/libprocess"
make[3]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess'
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty'
 (cd src && make  top_distdir=../mesos-0.22.0 distdir=../mesos-0.22.0/src \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/src'
test ".." = ".." ||			\
		(/bin/mkdir -p python/src/mesos && cp -pf ./python/src/mesos/__init__.py python/src/mesos/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos && cp -pf ./python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos/interface && cp -pf ./python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos && cp -pf ./python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/module.cpp python/native/src/mesos/native/module.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/module.hpp python/native/src/mesos/native/module.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_executor.cpp python/native/src/mesos/native/proxy_executor.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_executor.hpp python/native/src/mesos/native/proxy_executor.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_scheduler.cpp python/native/src/mesos/native/proxy_scheduler.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_scheduler.hpp python/native/src/mesos/native/proxy_scheduler.hpp)
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/src'
 (cd ec2 && make  top_distdir=../mesos-0.22.0 distdir=../mesos-0.22.0/ec2 \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/ec2'
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/ec2'
test -n "" \
	|| find "mesos-0.22.0" -type d ! -perm -755 \
		-exec chmod u+rwx,go+rx {} \; -o \
	  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -444 -exec /bin/bash /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/install-sh -c -m a+r {} {} \; \
	|| chmod -R a+r "mesos-0.22.0"
tardir=mesos-0.22.0 && ${TAR-tar} chof - "$tardir" | GZIP=--best gzip -c >mesos-0.22.0.tar.gz
make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot'
if test -d "mesos-0.22.0"; then find "mesos-0.22.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "mesos-0.22.0" || { sleep 5 && rm -rf "mesos-0.22.0"; }; else :; fi
case 'mesos-0.22.0.tar.gz' in \
	*.tar.gz*) \
	  GZIP=--best gzip -dc mesos-0.22.0.tar.gz | ${TAR-tar} xf - ;;\
	*.tar.bz2*) \
	  bzip2 -dc mesos-0.22.0.tar.bz2 | ${TAR-tar} xf - ;;\
	*.tar.lz*) \
	  lzip -dc mesos-0.22.0.tar.lz | ${TAR-tar} xf - ;;\
	*.tar.xz*) \
	  xz -dc mesos-0.22.0.tar.xz | ${TAR-tar} xf - ;;\
	*.tar.Z*) \
	  uncompress -c mesos-0.22.0.tar.Z | ${TAR-tar} xf - ;;\
	*.shar.gz*) \
	  GZIP=--best gzip -dc mesos-0.22.0.shar.gz | unshar ;;\
	*.zip*) \
	  unzip mesos-0.22.0.zip ;;\
	esac
chmod -R a-w mesos-0.22.0
chmod u+w mesos-0.22.0
mkdir mesos-0.22.0/_build mesos-0.22.0/_inst
chmod a-w mesos-0.22.0
test -d mesos-0.22.0/_build || exit 0; \
	dc_install_base=`CDPATH="${ZSH_VERSION+.}:" && cd mesos-0.22.0/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \
	  && dc_destdir="${TMPDIR-/tmp}/am-dc-$$/" \
	  && am__cwd=`pwd` \
	  && CDPATH="${ZSH_VERSION+.}:" && cd mesos-0.22.0/_build \
	  && ../configure \
	    --disable-python-dependency-install \
	     \
	    --srcdir=.. --prefix="$dc_install_base" \
	  && make  \
	  && make  dvi \
	  && make  check \
	  && make  install \
	  && make  installcheck \
	  && make  uninstall \
	  && make  distuninstallcheck_dir="$dc_install_base" \
	        distuninstallcheck \
	  && chmod -R a-w "$dc_install_base" \
	  && ({ \
	       (cd ../.. && umask 077 && mkdir "$dc_destdir") \
	       && make  DESTDIR="$dc_destdir" install \
	       && make  DESTDIR="$dc_destdir" uninstall \
	       && make  DESTDIR="$dc_destdir" \
	            distuninstallcheck_dir="$dc_destdir" distuninstallcheck; \
	      } || { rm -rf "$dc_destdir"; exit 1; }) \
	  && rm -rf "$dc_destdir" \
	  && make  dist \
	  && rm -rf mesos-0.22.0.tar.gz \
	  && make  distcleancheck \
	  && cd "$am__cwd" \
	  || exit 1
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
configure: creating ./config.lt
config.lt: creating libtool
configure: Setting up build environment for x86_64 linux-gnu
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking if compiling with clang... no
configure: GCC version: 4.8
configure: Setting up CXXFLAGS for g++ version >= 4.8
checking whether g++ supports C++11 features by default... no
checking whether g++ supports C++11 features with -std=c++11... yes
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for backtrace in -lunwind... no
checking for gzread in -lz... yes
checking for curl_global_init in -lcurl... yes
checking apr_pools.h usability... yes
checking apr_pools.h presence... yes
checking for apr_pools.h... yes
checking for apr_initialize in -lapr-1... yes
checking svn_version.h usability... yes
checking svn_version.h presence... yes
checking for svn_version.h... yes
checking for svn_stringbuf_create_ensure in -lsvn_subr-1... yes
checking svn_delta.h usability... yes
checking svn_delta.h presence... yes
checking for svn_delta.h... yes
checking for svn_txdelta in -lsvn_delta-1... yes
checking for sasl_done in -lsasl2... yes
checking whether or not we can build with JNI... yes
checking for mvn... /home/jenkins/tools/maven/latest/bin/mvn
checking for a Python interpreter with version >= 2.6... python
checking for python... /usr/bin/python
checking for python version... 2.7
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages
checking for python2.7... (cached) /usr/bin/python
checking for a version of Python >= '2.1.0'... yes
checking for a version of Python >= '2.6'... yes
checking for the distutils Python package... yes
checking for Python include path... -I/usr/include/python2.7
checking for Python library path... -L/usr/lib -lpython2.7
checking for Python site-packages path... /usr/lib/python2.7/dist-packages
checking python extra libraries... -lpthread -ldl  -lutil
checking python extra linking flags... -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
checking consistency of all components of python development environment... yes
checking whether we can build usable Python eggs... cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled by default]
yes
checking for an old installation of the Mesos egg (before 0.20.0)... no
checking python2 module: boto... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating mesos.pc
config.status: creating ec2/Makefile
config.status: creating src/Makefile
config.status: creating 3rdparty/Makefile
config.status: creating bin/mesos.sh
config.status: creating bin/mesos-local.sh
config.status: creating bin/mesos-master.sh
config.status: creating bin/mesos-slave.sh
config.status: creating bin/mesos-tests.sh
config.status: creating bin/mesos-local-flags.sh
config.status: creating bin/mesos-master-flags.sh
config.status: creating bin/mesos-slave-flags.sh
config.status: creating bin/mesos-tests-flags.sh
config.status: creating bin/gdb-mesos-local.sh
config.status: creating bin/gdb-mesos-master.sh
config.status: creating bin/gdb-mesos-slave.sh
config.status: creating bin/gdb-mesos-tests.sh
config.status: creating bin/lldb-mesos-local.sh
config.status: creating bin/lldb-mesos-master.sh
config.status: creating bin/lldb-mesos-slave.sh
config.status: creating bin/lldb-mesos-tests.sh
config.status: creating bin/valgrind-mesos-local.sh
config.status: creating bin/valgrind-mesos-slave.sh
config.status: creating bin/valgrind-mesos-master.sh
config.status: creating bin/valgrind-mesos-tests.sh
config.status: creating src/deploy/mesos-daemon.sh
config.status: creating src/deploy/mesos-start-cluster.sh
config.status: creating src/deploy/mesos-start-masters.sh
config.status: creating src/deploy/mesos-start-slaves.sh
config.status: creating src/deploy/mesos-stop-cluster.sh
config.status: creating src/deploy/mesos-stop-masters.sh
config.status: creating src/deploy/mesos-stop-slaves.sh
config.status: creating include/mesos/version.hpp
config.status: creating src/java/generated/org/apache/mesos/MesosNativeLibrary.java
config.status: creating mpi/mpiexec-mesos
config.status: creating src/examples/java/test-exception-framework
config.status: creating src/examples/java/test-executor
config.status: creating src/examples/java/test-framework
config.status: creating src/examples/java/test-multiple-executors-framework
config.status: creating src/examples/java/test-log
config.status: creating src/java/mesos.pom
config.status: creating src/examples/python/test-executor
config.status: creating src/examples/python/test-framework
config.status: creating src/examples/python/test-containerizer
config.status: creating src/python/setup.py
config.status: creating src/python/interface/setup.py
config.status: creating src/python/native/ext_modules.py
config.status: creating src/python/native/setup.py
config.status: executing depfiles commands
config.status: executing libtool commands
=== configuring in 3rdparty/libprocess (/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess)
configure: running /bin/bash ../../../3rdparty/libprocess/configure --disable-option-checking '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst'  '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' --cache-file=/dev/null --srcdir=../../../3rdparty/libprocess
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
configure: creating ./config.lt
config.lt: creating libtool
checking apr_pools.h usability... yes
checking apr_pools.h presence... yes
checking for apr_pools.h... yes
checking for apr_initialize in -lapr-1... yes
checking svn_version.h usability... yes
checking svn_version.h presence... yes
checking for svn_version.h... yes
checking for svn_stringbuf_create_ensure in -lsvn_subr-1... yes
checking svn_delta.h usability... yes
checking svn_delta.h presence... yes
checking for svn_delta.h... yes
checking for svn_txdelta in -lsvn_delta-1... yes
checking for curl_global_init in -lcurl... yes
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking if compiling with clang... no
configure: GCC version: 4.8
configure: Setting up CXXFLAGS for g++-4.8
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for deflate, gzread, gzwrite, inflate in -lz... yes
checking whether g++ supports C++11 features by default... no
checking whether g++ supports C++11 features with -std=c++11... yes
checking for dlopen in -ldl... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating include/Makefile
config.status: creating 3rdparty/Makefile
config.status: creating 3rdparty/gmock_sources.cc
config.status: executing depfiles commands
config.status: executing libtool commands
=== configuring in 3rdparty/stout (/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout)
configure: running /bin/bash ../../../../../3rdparty/libprocess/3rdparty/stout/configure --disable-option-checking '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst'  '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' --cache-file=/dev/null --srcdir=../../../../../3rdparty/libprocess/3rdparty/stout
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating include/Makefile
make[1]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
Making all in .
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
make[2]: Nothing to be done for `all-am'.
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
Making all in 3rdparty
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
make  all-recursive
make[3]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
Making all in libprocess
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
Making all in 3rdparty
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/ry-http-parser-1c3624a.tar.gz | tar xf -
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/gmock-1.6.0.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/ry-http-parser-1c3624a.patch || patch -d ry-http-parser-1c3624a -p1 <../../../../3rdparty/libprocess/3rdparty/ry-http-parser-1c3624a.patch
touch ry-http-parser-1c3624a-stamp
test ! -e ../../../../3rdparty/libprocess/3rdparty/gmock-1.6.0.patch || patch -d gmock-1.6.0 -p1 <../../../../3rdparty/libprocess/3rdparty/gmock-1.6.0.patch
touch gmock-1.6.0-stamp
cd .. && /bin/bash ./config.status 3rdparty/gmock_sources.cc
config.status: creating 3rdparty/gmock_sources.cc
make  all-recursive
make[6]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
Making all in stout
make[7]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
Making all in .
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
make[8]: Nothing to be done for `all-am'.
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
Making all in include
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout/include'
make[8]: Nothing to be done for `all'.
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout/include'
make[7]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
make[7]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../../3rdparty/libprocess/3rdparty  -Iry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -MT libry_http_parser_la-http_parser.lo -MD -MP -MF .deps/libry_http_parser_la-http_parser.Tpo -c -o libry_http_parser_la-htt
 p_parser.lo `test -f 'ry-http-parser-1c3624a/http_parser.c' || echo '../../../../3rdparty/libprocess/3rdparty/'`ry-http-parser-1c3624a/http_parser.c
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/protobuf-2.5.0.tar.gz | tar xf -
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/libev-4.15.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/libev-4.15.patch || patch -d libev-4.15 -p1 <../../../../3rdparty/libprocess/3rdparty/libev-4.15.patch
patching file ev.h
touch libev-4.15-stamp
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/boost-1.53.0.tar.gz | tar xf -
libtool: compile:  gcc -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../../3rdparty/libprocess/3rdparty -Iry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -g1 -O0 -Wno-unused-local-typedefs -MT libry_http_parser_la-http_parser.lo -MD -MP -MF .deps/libry_http_parser_la-http_parser.Tpo -c ry-http-parser-1c3624a/http_parser.c  -fPIC -DPIC -o libry
 _http_parser_la-http_parser.o
test ! -e ../../../../3rdparty/libprocess/3rdparty/protobuf-2.5.0.patch || patch -d protobuf-2.5.0 -p1 <../../../../3rdparty/libprocess/3rdparty/protobuf-2.5.0.patch
touch protobuf-2.5.0-stamp
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/picojson-4f93734.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/picojson-4f93734.patch || patch -d picojson-4f93734 -p1 <../../../../3rdparty/libprocess/3rdparty/picojson-4f93734.patch
touch picojson-4f93734-stamp
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/glog-0.3.3.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/glog-0.3.3.patch || patch -d glog-0.3.3 -p1 <../../../../3rdparty/libprocess/3rdparty/glog-0.3.3.patch
patching file src/symbolize.cc
patching file src/glog/stl_logging.h.in
patching file src/stl_logging_unittest.cc
patching file src/demangle.cc
touch glog-0.3.3-stamp
test ! -e ../../../../3rdparty/libprocess/3rdparty/boost-1.53.0.patch || patch -d boost-1.53.0 -p1 <../../../../3rdparty/libprocess/3rdparty/boost-1.53.0.patch
cd protobuf-2.5.0 && ./configure '--disable-option-checking' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' '--cache-file=/dev/null' '--srcdir=../../../3rdparty/libprocess' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
          make 
touch boost-1.53.0-stamp
cd libev-4.15 && ./configure '--disable-option-checking' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' '--cache-file=/dev/null' '--srcdir=../../../3rdparty/libprocess' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
          make 
mv -f .deps/libry_http_parser_la-http_parser.Tpo .deps/libry_http_parser_la-http_parser.Plo
cd glog-0.3.3 && ./configure GTEST_CONFIG=no '--disable-option-checking' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' '--cache-file=/dev/null' '--srcdir=../../../3rdparty/libprocess' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
          make 
checking whether to enable maintainer-specific portions of Makefiles... yes
checking for a BSD-compatible install... checking for a BSD-compatible install... checking build system type... /usr/bin/install -c
checking whether build environment is sane... /usr/bin/install -c
checking whether build environment is sane... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
yes
checking for a thread-safe mkdir -p... checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... /bin/mkdir -p
mawk
checking for gawk... checking whether make sets $(MAKE)... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
yes
checking for gcc... checking whether to enable maintainer-specific portions of Makefiles... no
gcc
checking for gcc... gcc
yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... checking whether the C compiler works... checking whether the C compiler works... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 

checking whether we are cross compiling... checking whether we are cross compiling... 
checking whether we are cross compiling... no
checking for suffix of object files... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... o
checking whether we are using the GNU C compiler... no
checking for suffix of object files... yes
checking whether gcc accepts -g... yes
checking whether gcc accepts -g... o
checking whether we are using the GNU C compiler... yes
checking for gcc option to accept ISO C89... yes
checking for gcc option to accept ISO C89... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
none needed
checking for style of include used by make... checking for style of include used by make... GNU
GNU
checking dependency style of gcc... checking dependency style of gcc... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc3
gcc3
checking for g++... g++
checking build system type... gcc -E
checking whether we are using the GNU C++ compiler... checking for g++... g++
x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... yes
checking whether g++ accepts -g... printf
checking for a sed that does not truncate output... checking whether we are using the GNU C++ compiler... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
yes
checking the name lister (/usr/bin/nm -B) interface... checking dependency style of g++... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... gcc3
checking how to run the C++ preprocessor... g++ -E
gcc3
@
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... checking build system type... checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... ok
checking for sysroot... no
BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... checking for mt... mt
pass_all
checking if mt is a manifest tool... checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... no
checking how to run the C preprocessor... yes
checking for sys/types.h... gcc -E
@
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... checking for ANSI C header files... yes
checking for sys/stat.h... yes
checking for stdlib.h... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking for ANSI C header files... yes
checking for string.h... yes
checking for sys/types.h... no
checking for sys/types.h... yes
yes
checking for memory.h... checking for sys/stat.h... yes
checking for sys/stat.h... yes
yes
checking for strings.h... yes
checking for stdlib.h... checking for stdlib.h... yes
yes
yes
checking for inttypes.h... checking for string.h... checking for string.h... yes
yes
yes
checking for stdint.h... checking for memory.h... checking for memory.h... yes
yes
checking for strings.h... yes
checking for unistd.h... checking for strings.h... yes
yes
checking for inttypes.h... checking minix/config.h usability... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for stdint.h... yes
no
checking minix/config.h presence... checking for unistd.h... yes
no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... checking for unistd.h... yes
checking for dlfcn.h... yes
checking C++ compiler flags...... use user-supplied: -g1 -O0 -Wno-unused-local-typedefs -std=c++11
yes
checking whether __SUNPRO_CC is declared... checking for dlfcn.h... yes
checking for objdir... .libs
no
checking how to print strings... printf
checking for a sed that does not truncate output... yes
checking for objdir... /bin/sed
.libs
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... checking if gcc supports -fno-rtti -fno-exceptions... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc static flag -static works... ok
checking for sysroot... no
yes
checking if gcc supports -c -o file.o... checking for mt... mt
checking if mt is a manifest tool... no
yes
checking if gcc supports -c -o file.o... checking for dlfcn.h... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
yes
checking dynamic linker characteristics... checking for objdir... .libs
GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking sys/inotify.h usability... g++ -E
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking sys/inotify.h presence... yes
checking for sys/inotify.h... yes
checking sys/epoll.h usability... yes
checking if gcc static flag -static works... checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking sys/epoll.h presence... yes
yes
checking for sys/epoll.h... yes
yes
checking if gcc supports -c -o file.o... checking sys/event.h usability... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
no
checking sys/event.h presence... no
checking for sys/event.h... no
yes
checking if g++ supports -c -o file.o... checking port.h usability... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) no
checking for stdint.h... (cached) yes
checking for sys/types.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for unistd.h... (cached) yes
checking syscall.h usability... checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
yes
checking syscall.h presence... yes
checking for syscall.h... yes
checking sys/syscall.h usability... checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... no
checking port.h presence... yes
checking sys/syscall.h presence... yes
checking for sys/syscall.h... yes
checking execinfo.h usability... no
checking for port.h... no
checking poll.h usability... yes
checking if g++ static flag -static works... yes
checking execinfo.h presence... yes
checking for execinfo.h... yes
yes
checking poll.h presence... checking libunwind.h usability... yes
checking for poll.h... yes
checking sys/select.h usability... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) yes
checking fcntl.h usability... yes
checking sys/select.h presence... yes
checking for sys/select.h... yes
checking sys/eventfd.h usability... no
checking libunwind.h presence... no
checking for libunwind.h... no
checking ucontext.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for inttypes.h... (cached) yes
yes
checking sys/eventfd.h presence... checking limits.h usability... yes
checking for sys/eventfd.h... yes
yes
checking ucontext.h presence... checking sys/signalfd.h usability... yes
checking for ucontext.h... yes
checking sys/utsname.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking for stdlib.h... (cached) yes
yes
checking sys/signalfd.h presence... checking for unistd.h... (cached) yes
checking for working memcmp... yes
yes
checking for sys/signalfd.h... checking sys/utsname.h presence... yes
checking for inotify_init... yes
checking for sys/utsname.h... yes
checking pwd.h usability... yes
yes
checking pwd.h presence... checking for epoll_ctl... yes
checking for pwd.h... yes
checking syslog.h usability... yes
checking for working strtod... yes
checking for kqueue... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking sys/time.h usability... yes
no
checking for ftruncate... checking for port_create... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking glob.h usability... yes
no
checking for poll... checking for memset... yes
checking glob.h presence... yes
checking for glob.h... yes
checking unwind.h usability... yes
checking for select... yes
checking for mkdir... yes
checking unwind.h presence... yes
checking for unwind.h... yes
checking size of void *... yes
checking for eventfd... yes
checking for strchr... yes
checking for signalfd... yes
8
checking for strerror... checking for uint16_t... yes
checking for clock_gettime... yes
checking for strtol... yes
checking for nanosleep... yes
checking for u_int16_t... yes
checking zlib version... yes
checking for library containing floor... ok (1.2.0.4 or later)
checking for library containing zlibVersion... yes
checking for __uint16... -lm
no
checking for sigaltstack... configure: creating ./config.status
-lz
checking for the pthreads library -lpthreads... yes
checking for dladdr... no
checking whether pthreads work without any flags... no
checking for fcntl... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... yes
checking for __attribute__... yes
checking for __builtin_expect... no
checking whether pthreads work with -pthread... yes
checking for __sync_val_compare_and_swap... yes
checking for joinable pthread attribute... yes
checking for the pthreads library -lpthreads... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... no
checking whether pthreads work without any flags... yes
checking whether what we have so far is sufficient with -nostdlib... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether -lpthread saves the day... no
checking whether pthreads work with -pthread... config.status: creating Makefile
yes
checking the location of hash_map... config.status: creating config.h
config.status: executing depfiles commands
yes
checking for joinable pthread attribute... config.status: executing libtool commands
PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for pthread_self in -lpthread... make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
make  all-am
make[9]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
/bin/bash ./libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.     -g -O3 -MT ev.lo -MD -MP -MF .deps/ev.Tpo -c -o ev.lo ev.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O3 -MT ev.lo -MD -MP -MF .deps/ev.Tpo -c ev.c  -fPIC -DPIC -o ev.o
yes
checking for main in -lgflags... no
checking for gtest-config... no
checking support for pthread_rwlock_* functions... ev.c:1531:31: warning: 'ev_default_loop_ptr' initialized and declared 'extern' [enabled by default]
   EV_API_DECL struct ev_loop *ev_default_loop_ptr = 0; /* needs to be initialised to make it a definition despite extern */
                               ^
yes
checking whether the compiler implements namespaces... ev.c: In function 'evpipe_write':
ev.c:2160:17: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result]
           write (evpipe [1], &counter, sizeof (uint64_t));
                 ^
ev.c:2172:17: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result]
           write (evpipe [1], &(evpipe [1]), 1);
                 ^
ev.c: In function 'pipecb':
ev.c:2193:16: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result]
           read (evpipe [1], &counter, sizeof (uint64_t));
                ^
ev.c:2207:16: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result]
           read (evpipe [0], &dummy, sizeof (dummy));
                ^
yes
checking what namespace STL code is in... <unordered_map>
configure: creating ./config.status
std
checking whether compiler supports using ::operator<<... 1
checking for ucontext.h... (cached) yes
checking sys/ucontext.h usability... yes
checking sys/ucontext.h presence... yes
checking for sys/ucontext.h... yes
checking how to access the program counter from a struct ucontext... uc_mcontext.gregs[REG_RIP]
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating protobuf.pc
configure: creating ./config.status
config.status: creating protobuf-lite.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
mv -f .deps/ev.Tpo .deps/ev.Plo
/bin/bash ./libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.     -g -O3 -MT event.lo -MD -MP -MF .deps/event.Tpo -c -o event.lo event.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O3 -MT event.lo -MD -MP -MF .deps/event.Tpo -c event.c  -fPIC -DPIC -o event.o
=== configuring in gtest (/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/gtest)
configure: running /bin/bash ./configure --disable-option-checking '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst'  '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' 'CXXFLAGS=-g1 -O0 -Wno-unused-local-typedefs -std=c++11' '--enable-shared=no' '--with-pic' --cache-file=/dev/null --srcdir=.
mv -f .deps/event.Tpo .deps/event.Plo
/bin/bash ./libtool --tag=CC   --mode=link gcc  -g -O3 -version-info 4:0:0  -o libev.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib ev.lo event.lo  -lm 
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... libtool: link: ar cru .libs/libev.a  ev.o event.o
libtool: link: ranlib .libs/libev.a
libtool: link: ( cd ".libs" && rm -f "libev.la" && ln -s "../libev.la" "libev.la" )
make[9]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g1 -O0 -Wno-unused-local-typedefs   -o libry_http_parser.la  libry_http_parser_la-http_parser.lo  -ldl -lz -lcurl -lsvn_delta-1 -lsvn_subr-1 -lapr-1  -lrt
config.status: creating Makefile
config.status: creating src/glog/logging.h
config.status: creating src/glog/raw_logging.h
config.status: creating src/glog/vlog_is_on.h
config.status: creating src/glog/stl_logging.h
config.status: creating libglog.pc
libtool: link: ar cru .libs/libry_http_parser.a  libry_http_parser_la-http_parser.o
libtool: link: ranlib .libs/libry_http_parser.a
config.status: creating src/config.h
libtool: link: ( cd ".libs" && rm -f "libry_http_parser.la" && ln -s "../libry_http_parser.la" "libry_http_parser.la" )
config.status: executing depfiles commands
config.status: executing libtool commands
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/glog-0.3.3'
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-logging.lo -MD -MP -MF .deps/libglog_la-logging.Tpo -c -o libglog_la-logging.lo `test -f 'src/logging.cc' || echo './'`src/logging.cc
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-raw_logging.lo -MD -MP -MF .deps/libglog_la-raw_logging.Tpo -c -o libglog_la-raw_logging.lo `test -f 'src/raw_logging.cc' || echo './'`src/raw_logging.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-raw_logging.lo -MD -MP -MF .deps/libglog_la-raw_logging.Tpo -c src/raw_logging.cc  -fPIC -DPIC -o libglog_la-raw_logging.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-logging.lo -MD -MP -MF .deps/libglog_la-logging.Tpo -c src/logging.cc  -fPIC -DPIC -o libglog_la-logging.o
mv -f .deps/libglog_la-raw_logging.Tpo .deps/libglog_la-raw_logging.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-vlog_is_on.lo -MD -MP -MF .deps/libglog_la-vlog_is_on.Tpo -c -o libglog_la-vlog_is_on.lo `test -f 'src/vlog_is_on.cc' || echo './'`src/vlog_is_on.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-vlog_is_on.lo -MD -MP -MF .deps/libglog_la-vlog_is_on.Tpo -c src/vlog_is_on.cc  -fPIC -DPIC -o libglog_la-vlog_is_on.o
yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for g++... g++
mv -f .deps/libglog_la-vlog_is_on.Tpo .deps/libglog_la-vlog_is_on.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-utilities.lo -MD -MP -MF .deps/libglog_la-utilities.Tpo -c -o libglog_la-utilities.lo `test -f 'src/utilities.cc' || echo './'`src/utilities.cc
checking whether we are using the GNU C++ compiler... libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-utilities.lo -MD -MP -MF .deps/libglog_la-utilities.Tpo -c src/utilities.cc  -fPIC -DPIC -o libglog_la-utilities.o
yes
checking whether g++ accepts -g... yes
checking dependency style of g++... mv -f .deps/libglog_la-logging.Tpo .deps/libglog_la-logging.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-demangle.lo -MD -MP -MF .deps/libglog_la-demangle.Tpo -c -o libglog_la-demangle.lo `test -f 'src/demangle.cc' || echo './'`src/demangle.cc
gcc3
checking build system type... libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-demangle.lo -MD -MP -MF .deps/libglog_la-demangle.Tpo -c src/demangle.cc  -fPIC -DPIC -o libglog_la-demangle.o
x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... mv -f .deps/libglog_la-demangle.Tpo .deps/libglog_la-demangle.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-symbolize.lo -MD -MP -MF .deps/libglog_la-symbolize.Tpo -c -o libglog_la-symbolize.lo `test -f 'src/symbolize.cc' || echo './'`src/symbolize.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-symbolize.lo -MD -MP -MF .deps/libglog_la-symbolize.Tpo -c src/symbolize.cc  -fPIC -DPIC -o libglog_la-symbolize.o
ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... mv -f .deps/libglog_la-utilities.Tpo .deps/libglog_la-utilities.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-signalhandler.lo -MD -MP -MF .deps/libglog_la-signalhandler.Tpo -c -o libglog_la-signalhandler.lo `test -f 'src/signalhandler.cc' || echo './'`src/signalhandler.cc
gcc -E
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-signalhandler.lo -MD -MP -MF .deps/libglog_la-signalhandler.Tpo -c src/signalhandler.cc  -fPIC -DPIC -o libglog_la-signalhandler.o
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... mv -f .deps/libglog_la-symbolize.Tpo .deps/libglog_la-symbolize.Plo
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_unittest-logging_unittest.o -MD -MP -MF .deps/logging_unittest-logging_unittest.Tpo -c -o logging_unittest-logging_unittest.o `test -f 'src/logging_unittest.cc' || echo './'`src/logging_unittest.cc
yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... mv -f .deps/libglog_la-signalhandler.Tpo .deps/libglog_la-signalhandler.Plo
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT demangle_unittest-demangle_unittest.o -MD -MP -MF .deps/demangle_unittest-demangle_unittest.Tpo -c -o demangle_unittest-demangle_unittest.o `test -f 'src/demangle_unittest.cc' || echo './'`src/demangle_unittest.cc
yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for python... /usr/bin/python
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... mv -f .deps/demangle_unittest-demangle_unittest.Tpo .deps/demangle_unittest-demangle_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stacktrace_unittest-stacktrace_unittest.o -MD -MP -MF .deps/stacktrace_unittest-stacktrace_unittest.Tpo -c -o stacktrace_unittest-stacktrace_unittest.o `test -f 'src/stacktrace_unittest.cc' || echo './'`src/stacktrace_unittest.cc
no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
configure: creating ./config.status
mv -f .deps/stacktrace_unittest-stacktrace_unittest.Tpo .deps/stacktrace_unittest-stacktrace_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT symbolize_unittest-symbolize_unittest.o -MD -MP -MF .deps/symbolize_unittest-symbolize_unittest.Tpo -c -o symbolize_unittest-symbolize_unittest.o `test -f 'src/symbolize_unittest.cc' || echo './'`src/symbolize_unittest.cc
mv -f .deps/logging_unittest-logging_unittest.Tpo .deps/logging_unittest-logging_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stl_logging_unittest-stl_logging_unittest.o -MD -MP -MF .deps/stl_logging_unittest-stl_logging_unittest.Tpo -c -o stl_logging_unittest-stl_logging_unittest.o `test -f 'src/stl_logging_unittest.cc' || echo './'`src/stl_logging_unittest.cc
In file included from /usr/include/c++/4.8/ext/hash_set:60:0,
                 from src/glog/stl_logging.h:54,
                 from src/stl_logging_unittest.cc:34:
/usr/include/c++/4.8/backward/backward_warning.h:32:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp]
 #warning \
  ^
In file included from src/utilities.h:73:0,
                 from src/googletest.h:38,
                 from src/stl_logging_unittest.cc:48:
src/base/mutex.h:137:0: warning: "_XOPEN_SOURCE" redefined [enabled by default]
 #   define _XOPEN_SOURCE 500  // may be needed to get the rwlock calls
 ^
In file included from /usr/include/x86_64-linux-gnu/c++/4.8/bits/os_defines.h:39:0,
                 from /usr/include/x86_64-linux-gnu/c++/4.8/bits/c++config.h:426,
                 from /usr/include/c++/4.8/bits/stl_algobase.h:59,
                 from /usr/include/c++/4.8/deque:60,
                 from src/glog/stl_logging.h:45,
                 from src/stl_logging_unittest.cc:34:
/usr/include/features.h:162:0: note: this is the location of the previous definition
 # define _XOPEN_SOURCE 700
 ^
config.status: creating Makefile
config.status: creating scripts/gtest-config
config.status: creating build-aux/config.h
config.status: executing depfiles commands
config.status: executing libtool commands
mv -f .deps/symbolize_unittest-symbolize_unittest.Tpo .deps/symbolize_unittest-symbolize_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT utilities_unittest-utilities_unittest.o -MD -MP -MF .deps/utilities_unittest-utilities_unittest.Tpo -c -o utilities_unittest-utilities_unittest.o `test -f 'src/utilities_unittest.cc' || echo './'`src/utilities_unittest.cc
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
make  all-recursive
make[9]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
Making all in .
make[10]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
make[10]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
Making all in src
make[10]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o `test -f 'google/protobuf/compiler/main.cc' || echo './'`google/protobuf/compiler/main.cc
mv -f .deps/main.Tpo .deps/main.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_gcc.lo -MD -MP -MF .deps/atomicops_internals_x86_gcc.Tpo -c -o atomicops_internals_x86_gcc.lo `test -f 'google/protobuf/stubs/atomicops_internals_x86_gcc.cc' || echo './'`google/protobuf/stubs/atomicops_internals_x86_gcc.cc
mv -f .deps/utilities_unittest-utilities_unittest.Tpo .deps/utilities_unittest-utilities_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_striptest0-logging_striptest_main.o -MD -MP -MF .deps/logging_striptest0-logging_striptest_main.Tpo -c -o logging_striptest0-logging_striptest_main.o `test -f 'src/logging_striptest_main.cc' || echo './'`src/logging_striptest_main.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_gcc.lo -MD -MP -MF .deps/atomicops_internals_x86_gcc.Tpo -c google/protobuf/stubs/atomicops_internals_x86_gcc.cc  -fPIC -DPIC -o atomicops_internals_x86_gcc.o
mv -f .deps/atomicops_internals_x86_gcc.Tpo .deps/atomicops_internals_x86_gcc.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_msvc.lo -MD -MP -MF .deps/atomicops_internals_x86_msvc.Tpo -c -o atomicops_internals_x86_msvc.lo `test -f 'google/protobuf/stubs/atomicops_internals_x86_msvc.cc' || echo './'`google/protobuf/stubs/atomicops_internals_x86_msvc.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_msvc.lo -MD -MP -MF .deps/atomicops_internals_x86_msvc.Tpo -c google/protobuf/stubs/atomicops_internals_x86_msvc.cc  -fPIC -DPIC -o atomicops_internals_x86_msvc.o
src/logging_striptest_main.cc: In function 'int CheckNoReturn(bool)':
src/logging_striptest_main.cc:54:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
mv -f .deps/logging_striptest0-logging_striptest_main.Tpo .deps/logging_striptest0-logging_striptest_main.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_striptest2-logging_striptest2.o -MD -MP -MF .deps/logging_striptest2-logging_striptest2.Tpo -c -o logging_striptest2-logging_striptest2.o `test -f 'src/logging_striptest2.cc' || echo './'`src/logging_striptest2.cc
mv -f .deps/atomicops_internals_x86_msvc.Tpo .deps/atomicops_internals_x86_msvc.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common.lo -MD -MP -MF .deps/common.Tpo -c -o common.lo `test -f 'google/protobuf/stubs/common.cc' || echo './'`google/protobuf/stubs/common.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common.lo -MD -MP -MF .deps/common.Tpo -c google/protobuf/stubs/common.cc  -fPIC -DPIC -o common.o
mv -f .deps/stl_logging_unittest-stl_logging_unittest.Tpo .deps/stl_logging_unittest-stl_logging_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_striptest10-logging_striptest10.o -MD -MP -MF .deps/logging_striptest10-logging_striptest10.Tpo -c -o logging_striptest10-logging_striptest10.o `test -f 'src/logging_striptest10.cc' || echo './'`src/logging_striptest10.cc
In file included from src/logging_striptest2.cc:35:0:
src/logging_striptest_main.cc: In function 'int CheckNoReturn(bool)':
src/logging_striptest_main.cc:54:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
mv -f .deps/logging_striptest2-logging_striptest2.Tpo .deps/logging_striptest2-logging_striptest2.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT signalhandler_unittest-signalhandler_unittest.o -MD -MP -MF .deps/signalhandler_unittest-signalhandler_unittest.Tpo -c -o signalhandler_unittest-signalhandler_unittest.o `test -f 'src/signalhandler_unittest.cc' || echo './'`src/signalhandler_unittest.cc
mv -f .deps/common.Tpo .deps/common.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT once.lo -MD -MP -MF .deps/once.Tpo -c -o once.lo `test -f 'google/protobuf/stubs/once.cc' || echo './'`google/protobuf/stubs/once.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT once.lo -MD -MP -MF .deps/once.Tpo -c google/protobuf/stubs/once.cc  -fPIC -DPIC -o once.o
In file included from src/logging_striptest10.cc:35:0:
src/logging_striptest_main.cc: In function 'int CheckNoReturn(bool)':
src/logging_striptest_main.cc:54:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
mv -f .deps/logging_striptest10-logging_striptest10.Tpo .deps/logging_striptest10-logging_striptest10.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stringprintf.lo -MD -MP -MF .deps/stringprintf.Tpo -c -o stringprintf.lo `test -f 'google/protobuf/stubs/stringprintf.cc' || echo './'`google/protobuf/stubs/stringprintf.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stringprintf.lo -MD -MP -MF .deps/stringprintf.Tpo -c google/protobuf/stubs/stringprintf.cc  -fPIC -DPIC -o stringprintf.o
mv -f .deps/signalhandler_unittest-signalhandler_unittest.Tpo .deps/signalhandler_unittest-signalhandler_unittest.Po
/bin/bash ./libtool --tag=CXX   --mode=link g++   -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11    -o libglog.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib  libglog_la-logging.lo libglog_la-raw_logging.lo libglog_la-vlog_is_on.lo libglog_la-utilities.lo libglog_la-demangle.lo libglog_la-symbolize.lo libglog_la-signalhandler.lo     -lpthread 
mv -f .deps/once.Tpo .deps/once.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set.lo -MD -MP -MF .deps/extension_set.Tpo -c -o extension_set.lo `test -f 'google/protobuf/extension_set.cc' || echo './'`google/protobuf/extension_set.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set.lo -MD -MP -MF .deps/extension_set.Tpo -c google/protobuf/extension_set.cc  -fPIC -DPIC -o extension_set.o
libtool: link: ar cru .libs/libglog.a  libglog_la-logging.o libglog_la-raw_logging.o libglog_la-vlog_is_on.o libglog_la-utilities.o libglog_la-demangle.o libglog_la-symbolize.o libglog_la-signalhandler.o
libtool: link: ranlib .libs/libglog.a
libtool: link: ( cd ".libs" && rm -f "libglog.la" && ln -s "../libglog.la" "libglog.la" )
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_unittest  logging_unittest-logging_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_unittest logging_unittest-logging_unittest.o  ./.libs/libglog.a -lpthread -pthread
mv -f .deps/stringprintf.Tpo .deps/stringprintf.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_util.lo -MD -MP -MF .deps/generated_message_util.Tpo -c -o generated_message_util.lo `test -f 'google/protobuf/generated_message_util.cc' || echo './'`google/protobuf/generated_message_util.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_util.lo -MD -MP -MF .deps/generated_message_util.Tpo -c google/protobuf/generated_message_util.cc  -fPIC -DPIC -o generated_message_util.o
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o demangle_unittest  demangle_unittest-demangle_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o demangle_unittest demangle_unittest-demangle_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o stacktrace_unittest  stacktrace_unittest-stacktrace_unittest.o  libglog.la    -lpthread 
mv -f .deps/generated_message_util.Tpo .deps/generated_message_util.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message_lite.lo -MD -MP -MF .deps/message_lite.Tpo -c -o message_lite.lo `test -f 'google/protobuf/message_lite.cc' || echo './'`google/protobuf/message_lite.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message_lite.lo -MD -MP -MF .deps/message_lite.Tpo -c google/protobuf/message_lite.cc  -fPIC -DPIC -o message_lite.o
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o stacktrace_unittest stacktrace_unittest-stacktrace_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o symbolize_unittest  symbolize_unittest-symbolize_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o symbolize_unittest symbolize_unittest-symbolize_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o stl_logging_unittest  stl_logging_unittest-stl_logging_unittest.o  libglog.la       -lpthread 
mv -f .deps/message_lite.Tpo .deps/message_lite.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT repeated_field.lo -MD -MP -MF .deps/repeated_field.Tpo -c -o repeated_field.lo `test -f 'google/protobuf/repeated_field.cc' || echo './'`google/protobuf/repeated_field.cc
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o stl_logging_unittest stl_logging_unittest-stl_logging_unittest.o  ./.libs/libglog.a -lpthread -pthread
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT repeated_field.lo -MD -MP -MF .deps/repeated_field.Tpo -c google/protobuf/repeated_field.cc  -fPIC -DPIC -o repeated_field.o
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o utilities_unittest  utilities_unittest-utilities_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o utilities_unittest utilities_unittest-utilities_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_striptest0  logging_striptest0-logging_striptest_main.o  libglog.la    -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_striptest0 logging_striptest0-logging_striptest_main.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_striptest2  logging_striptest2-logging_striptest2.o  libglog.la    -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_striptest2 logging_striptest2-logging_striptest2.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_striptest10  logging_striptest10-logging_striptest10.o  libglog.la    -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_striptest10 logging_striptest10-logging_striptest10.o  ./.libs/libglog.a -lpthread -pthread
mv -f .deps/repeated_field.Tpo .deps/repeated_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format_lite.lo -MD -MP -MF .deps/wire_format_lite.Tpo -c -o wire_format_lite.lo `test -f 'google/protobuf/wire_format_lite.cc' || echo './'`google/protobuf/wire_format_lite.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format_lite.lo -MD -MP -MF .deps/wire_format_lite.Tpo -c google/protobuf/wire_format_lite.cc  -fPIC -DPIC -o wire_format_lite.o
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o signalhandler_unittest  signalhandler_unittest-signalhandler_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o signalhandler_unittest signalhandler_unittest-signalhandler_unittest.o  ./.libs/libglog.a -lpthread -pthread
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/glog-0.3.3'
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT coded_stream.lo -MD -MP -MF .deps/coded_stream.Tpo -c -o coded_stream.lo `test -f 'google/protobuf/io/coded_stream.cc' || echo './'`google/protobuf/io/coded_stream.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT coded_stream.lo -MD -MP -MF .deps/coded_stream.Tpo -c google/protobuf/io/coded_stream.cc  -fPIC -DPIC -o coded_stream.o
mv -f .deps/extension_set.Tpo .deps/extension_set.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream.lo -MD -MP -MF .deps/zero_copy_stream.Tpo -c -o zero_copy_stream.lo `test -f 'google/protobuf/io/zero_copy_stream.cc' || echo './'`google/protobuf/io/zero_copy_stream.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream.lo -MD -MP -MF .deps/zero_copy_stream.Tpo -c google/protobuf/io/zero_copy_stream.cc  -fPIC -DPIC -o zero_copy_stream.o
mv -f .deps/zero_copy_stream.Tpo .deps/zero_copy_stream.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl_lite.lo -MD -MP -MF .deps/zero_copy_stream_impl_lite.Tpo -c -o zero_copy_stream_impl_lite.lo `test -f 'google/protobuf/io/zero_copy_stream_impl_lite.cc' || echo './'`google/protobuf/io/zero_copy_stream_impl_lite.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl_lite.lo -MD -MP -MF .deps/zero_copy_stream_impl_lite.Tpo -c google/protobuf/io/zero_copy_stream_impl_lite.cc  -fPIC -DPIC -o zero_copy_stream_impl_lite.o
mv -f .deps/wire_format_lite.Tpo .deps/wire_format_lite.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT strutil.lo -MD -MP -MF .deps/strutil.Tpo -c -o strutil.lo `test -f 'google/protobuf/stubs/strutil.cc' || echo './'`google/protobuf/stubs/strutil.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT strutil.lo -MD -MP -MF .deps/strutil.Tpo -c google/protobuf/stubs/strutil.cc  -fPIC -DPIC -o strutil.o
mv -f .deps/coded_stream.Tpo .deps/coded_stream.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT substitute.lo -MD -MP -MF .deps/substitute.Tpo -c -o substitute.lo `test -f 'google/protobuf/stubs/substitute.cc' || echo './'`google/protobuf/stubs/substitute.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT substitute.lo -MD -MP -MF .deps/substitute.Tpo -c google/protobuf/stubs/substitute.cc  -fPIC -DPIC -o substitute.o
mv -f .deps/zero_copy_stream_impl_lite.Tpo .deps/zero_copy_stream_impl_lite.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT structurally_valid.lo -MD -MP -MF .deps/structurally_valid.Tpo -c -o structurally_valid.lo `test -f 'google/protobuf/stubs/structurally_valid.cc' || echo './'`google/protobuf/stubs/structurally_valid.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT structurally_valid.lo -MD -MP -MF .deps/structurally_valid.Tpo -c google/protobuf/stubs/structurally_valid.cc  -fPIC -DPIC -o structurally_valid.o
mv -f .deps/substitute.Tpo .deps/substitute.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.lo -MD -MP -MF .deps/descriptor.Tpo -c -o descriptor.lo `test -f 'google/protobuf/descriptor.cc' || echo './'`google/protobuf/descriptor.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.lo -MD -MP -MF .deps/descriptor.Tpo -c google/protobuf/descriptor.cc  -fPIC -DPIC -o descriptor.o
mv -f .deps/structurally_valid.Tpo .deps/structurally_valid.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.pb.lo -MD -MP -MF .deps/descriptor.pb.Tpo -c -o descriptor.pb.lo `test -f 'google/protobuf/descriptor.pb.cc' || echo './'`google/protobuf/descriptor.pb.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.pb.lo -MD -MP -MF .deps/descriptor.pb.Tpo -c google/protobuf/descriptor.pb.cc  -fPIC -DPIC -o descriptor.pb.o
mv -f .deps/strutil.Tpo .deps/strutil.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor_database.lo -MD -MP -MF .deps/descriptor_database.Tpo -c -o descriptor_database.lo `test -f 'google/protobuf/descriptor_database.cc' || echo './'`google/protobuf/descriptor_database.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor_database.lo -MD -MP -MF .deps/descriptor_database.Tpo -c google/protobuf/descriptor_database.cc  -fPIC -DPIC -o descriptor_database.o
mv -f .deps/descriptor_database.Tpo .deps/descriptor_database.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT dynamic_message.lo -MD -MP -MF .deps/dynamic_message.Tpo -c -o dynamic_message.lo `test -f 'google/protobuf/dynamic_message.cc' || echo './'`google/protobuf/dynamic_message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT dynamic_message.lo -MD -MP -MF .deps/dynamic_message.Tpo -c google/protobuf/dynamic_message.cc  -fPIC -DPIC -o dynamic_message.o
mv -f .deps/descriptor.pb.Tpo .deps/descriptor.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set_heavy.lo -MD -MP -MF .deps/extension_set_heavy.Tpo -c -o extension_set_heavy.lo `test -f 'google/protobuf/extension_set_heavy.cc' || echo './'`google/protobuf/extension_set_heavy.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set_heavy.lo -MD -MP -MF .deps/extension_set_heavy.Tpo -c google/protobuf/extension_set_heavy.cc  -fPIC -DPIC -o extension_set_heavy.o
mv -f .deps/dynamic_message.Tpo .deps/dynamic_message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_reflection.lo -MD -MP -MF .deps/generated_message_reflection.Tpo -c -o generated_message_reflection.lo `test -f 'google/protobuf/generated_message_reflection.cc' || echo './'`google/protobuf/generated_message_reflection.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_reflection.lo -MD -MP -MF .deps/generated_message_reflection.Tpo -c google/protobuf/generated_message_reflection.cc  -fPIC -DPIC -o generated_message_reflection.o
mv -f .deps/extension_set_heavy.Tpo .deps/extension_set_heavy.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message.lo -MD -MP -MF .deps/message.Tpo -c -o message.lo `test -f 'google/protobuf/message.cc' || echo './'`google/protobuf/message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message.lo -MD -MP -MF .deps/message.Tpo -c google/protobuf/message.cc  -fPIC -DPIC -o message.o
mv -f .deps/descriptor.Tpo .deps/descriptor.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT reflection_ops.lo -MD -MP -MF .deps/reflection_ops.Tpo -c -o reflection_ops.lo `test -f 'google/protobuf/reflection_ops.cc' || echo './'`google/protobuf/reflection_ops.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT reflection_ops.lo -MD -MP -MF .deps/reflection_ops.Tpo -c google/protobuf/reflection_ops.cc  -fPIC -DPIC -o reflection_ops.o
mv -f .deps/generated_message_reflection.Tpo .deps/generated_message_reflection.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT service.lo -MD -MP -MF .deps/service.Tpo -c -o service.lo `test -f 'google/protobuf/service.cc' || echo './'`google/protobuf/service.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT service.lo -MD -MP -MF .deps/service.Tpo -c google/protobuf/service.cc  -fPIC -DPIC -o service.o
mv -f .deps/service.Tpo .deps/service.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT text_format.lo -MD -MP -MF .deps/text_format.Tpo -c -o text_format.lo `test -f 'google/protobuf/text_format.cc' || echo './'`google/protobuf/text_format.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT text_format.lo -MD -MP -MF .deps/text_format.Tpo -c google/protobuf/text_format.cc  -fPIC -DPIC -o text_format.o
mv -f .deps/message.Tpo .deps/message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT unknown_field_set.lo -MD -MP -MF .deps/unknown_field_set.Tpo -c -o unknown_field_set.lo `test -f 'google/protobuf/unknown_field_set.cc' || echo './'`google/protobuf/unknown_field_set.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT unknown_field_set.lo -MD -MP -MF .deps/unknown_field_set.Tpo -c google/protobuf/unknown_field_set.cc  -fPIC -DPIC -o unknown_field_set.o
mv -f .deps/reflection_ops.Tpo .deps/reflection_ops.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format.lo -MD -MP -MF .deps/wire_format.Tpo -c -o wire_format.lo `test -f 'google/protobuf/wire_format.cc' || echo './'`google/protobuf/wire_format.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format.lo -MD -MP -MF .deps/wire_format.Tpo -c google/protobuf/wire_format.cc  -fPIC -DPIC -o wire_format.o
mv -f .deps/unknown_field_set.Tpo .deps/unknown_field_set.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT gzip_stream.lo -MD -MP -MF .deps/gzip_stream.Tpo -c -o gzip_stream.lo `test -f 'google/protobuf/io/gzip_stream.cc' || echo './'`google/protobuf/io/gzip_stream.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT gzip_stream.lo -MD -MP -MF .deps/gzip_stream.Tpo -c google/protobuf/io/gzip_stream.cc  -fPIC -DPIC -o gzip_stream.o
mv -f .deps/text_format.Tpo .deps/text_format.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT printer.lo -MD -MP -MF .deps/printer.Tpo -c -o printer.lo `test -f 'google/protobuf/io/printer.cc' || echo './'`google/protobuf/io/printer.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT printer.lo -MD -MP -MF .deps/printer.Tpo -c google/protobuf/io/printer.cc  -fPIC -DPIC -o printer.o
mv -f .deps/wire_format.Tpo .deps/wire_format.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT tokenizer.lo -MD -MP -MF .deps/tokenizer.Tpo -c -o tokenizer.lo `test -f 'google/protobuf/io/tokenizer.cc' || echo './'`google/protobuf/io/tokenizer.cc
mv -f .deps/gzip_stream.Tpo .deps/gzip_stream.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl.lo -MD -MP -MF .deps/zero_copy_stream_impl.Tpo -c -o zero_copy_stream_impl.lo `test -f 'google/protobuf/io/zero_copy_stream_impl.cc' || echo './'`google/protobuf/io/zero_copy_stream_impl.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT tokenizer.lo -MD -MP -MF .deps/tokenizer.Tpo -c google/protobuf/io/tokenizer.cc  -fPIC -DPIC -o tokenizer.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl.lo -MD -MP -MF .deps/zero_copy_stream_impl.Tpo -c google/protobuf/io/zero_copy_stream_impl.cc  -fPIC -DPIC -o zero_copy_stream_impl.o
mv -f .deps/printer.Tpo .deps/printer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT importer.lo -MD -MP -MF .deps/importer.Tpo -c -o importer.lo `test -f 'google/protobuf/compiler/importer.cc' || echo './'`google/protobuf/compiler/importer.cc
mv -f .deps/tokenizer.Tpo .deps/tokenizer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT parser.lo -MD -MP -MF .deps/parser.Tpo -c -o parser.lo `test -f 'google/protobuf/compiler/parser.cc' || echo './'`google/protobuf/compiler/parser.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT importer.lo -MD -MP -MF .deps/importer.Tpo -c google/protobuf/compiler/importer.cc  -fPIC -DPIC -o importer.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT parser.lo -MD -MP -MF .deps/parser.Tpo -c google/protobuf/compiler/parser.cc  -fPIC -DPIC -o parser.o
mv -f .deps/zero_copy_stream_impl.Tpo .deps/zero_copy_stream_impl.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT code_generator.lo -MD -MP -MF .deps/code_generator.Tpo -c -o code_generator.lo `test -f 'google/protobuf/compiler/code_generator.cc' || echo './'`google/protobuf/compiler/code_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT code_generator.lo -MD -MP -MF .deps/code_generator.Tpo -c google/protobuf/compiler/code_generator.cc  -fPIC -DPIC -o code_generator.o
mv -f .deps/code_generator.Tpo .deps/code_generator.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT command_line_interface.lo -MD -MP -MF .deps/command_line_interface.Tpo -c -o command_line_interface.lo `test -f 'google/protobuf/compiler/command_line_interface.cc' || echo './'`google/protobuf/compiler/command_line_interface.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT command_line_interface.lo -MD -MP -MF .deps/command_line_interface.Tpo -c google/protobuf/compiler/command_line_interface.cc  -fPIC -DPIC -o command_line_interface.o
mv -f .deps/importer.Tpo .deps/importer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.lo -MD -MP -MF .deps/plugin.Tpo -c -o plugin.lo `test -f 'google/protobuf/compiler/plugin.cc' || echo './'`google/protobuf/compiler/plugin.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.lo -MD -MP -MF .deps/plugin.Tpo -c google/protobuf/compiler/plugin.cc  -fPIC -DPIC -o plugin.o
mv -f .deps/parser.Tpo .deps/parser.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.pb.lo -MD -MP -MF .deps/plugin.pb.Tpo -c -o plugin.pb.lo `test -f 'google/protobuf/compiler/plugin.pb.cc' || echo './'`google/protobuf/compiler/plugin.pb.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.pb.lo -MD -MP -MF .deps/plugin.pb.Tpo -c google/protobuf/compiler/plugin.pb.cc  -fPIC -DPIC -o plugin.pb.o
mv -f .deps/plugin.Tpo .deps/plugin.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT subprocess.lo -MD -MP -MF .deps/subprocess.Tpo -c -o subprocess.lo `test -f 'google/protobuf/compiler/subprocess.cc' || echo './'`google/protobuf/compiler/subprocess.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT subprocess.lo -MD -MP -MF .deps/subprocess.Tpo -c google/protobuf/compiler/subprocess.cc  -fPIC -DPIC -o subprocess.o
mv -f .deps/command_line_interface.Tpo .deps/command_line_interface.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zip_writer.lo -MD -MP -MF .deps/zip_writer.Tpo -c -o zip_writer.lo `test -f 'google/protobuf/compiler/zip_writer.cc' || echo './'`google/protobuf/compiler/zip_writer.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zip_writer.lo -MD -MP -MF .deps/zip_writer.Tpo -c google/protobuf/compiler/zip_writer.cc  -fPIC -DPIC -o zip_writer.o
mv -f .deps/plugin.pb.Tpo .deps/plugin.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum.lo -MD -MP -MF .deps/cpp_enum.Tpo -c -o cpp_enum.lo `test -f 'google/protobuf/compiler/cpp/cpp_enum.cc' || echo './'`google/protobuf/compiler/cpp/cpp_enum.cc
mv -f .deps/subprocess.Tpo .deps/subprocess.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum_field.lo -MD -MP -MF .deps/cpp_enum_field.Tpo -c -o cpp_enum_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_enum_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_enum_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum.lo -MD -MP -MF .deps/cpp_enum.Tpo -c google/protobuf/compiler/cpp/cpp_enum.cc  -fPIC -DPIC -o cpp_enum.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum_field.lo -MD -MP -MF .deps/cpp_enum_field.Tpo -c google/protobuf/compiler/cpp/cpp_enum_field.cc  -fPIC -DPIC -o cpp_enum_field.o
mv -f .deps/zip_writer.Tpo .deps/zip_writer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_extension.lo -MD -MP -MF .deps/cpp_extension.Tpo -c -o cpp_extension.lo `test -f 'google/protobuf/compiler/cpp/cpp_extension.cc' || echo './'`google/protobuf/compiler/cpp/cpp_extension.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_extension.lo -MD -MP -MF .deps/cpp_extension.Tpo -c google/protobuf/compiler/cpp/cpp_extension.cc  -fPIC -DPIC -o cpp_extension.o
mv -f .deps/cpp_enum_field.Tpo .deps/cpp_enum_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_field.lo -MD -MP -MF .deps/cpp_field.Tpo -c -o cpp_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_field.lo -MD -MP -MF .deps/cpp_field.Tpo -c google/protobuf/compiler/cpp/cpp_field.cc  -fPIC -DPIC -o cpp_field.o
mv -f .deps/cpp_enum.Tpo .deps/cpp_enum.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_file.lo -MD -MP -MF .deps/cpp_file.Tpo -c -o cpp_file.lo `test -f 'google/protobuf/compiler/cpp/cpp_file.cc' || echo './'`google/protobuf/compiler/cpp/cpp_file.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_file.lo -MD -MP -MF .deps/cpp_file.Tpo -c google/protobuf/compiler/cpp/cpp_file.cc  -fPIC -DPIC -o cpp_file.o
mv -f .deps/cpp_extension.Tpo .deps/cpp_extension.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_generator.lo -MD -MP -MF .deps/cpp_generator.Tpo -c -o cpp_generator.lo `test -f 'google/protobuf/compiler/cpp/cpp_generator.cc' || echo './'`google/protobuf/compiler/cpp/cpp_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_generator.lo -MD -MP -MF .deps/cpp_generator.Tpo -c google/protobuf/compiler/cpp/cpp_generator.cc  -fPIC -DPIC -o cpp_generator.o
mv -f .deps/cpp_field.Tpo .deps/cpp_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_helpers.lo -MD -MP -MF .deps/cpp_helpers.Tpo -c -o cpp_helpers.lo `test -f 'google/protobuf/compiler/cpp/cpp_helpers.cc' || echo './'`google/protobuf/compiler/cpp/cpp_helpers.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_helpers.lo -MD -MP -MF .deps/cpp_helpers.Tpo -c google/protobuf/compiler/cpp/cpp_helpers.cc  -fPIC -DPIC -o cpp_helpers.o
mv -f .deps/cpp_generator.Tpo .deps/cpp_generator.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message.lo -MD -MP -MF .deps/cpp_message.Tpo -c -o cpp_message.lo `test -f 'google/protobuf/compiler/cpp/cpp_message.cc' || echo './'`google/protobuf/compiler/cpp/cpp_message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message.lo -MD -MP -MF .deps/cpp_message.Tpo -c google/protobuf/compiler/cpp/cpp_message.cc  -fPIC -DPIC -o cpp_message.o
mv -f .deps/cpp_file.Tpo .deps/cpp_file.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message_field.lo -MD -MP -MF .deps/cpp_message_field.Tpo -c -o cpp_message_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_message_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_message_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message_field.lo -MD -MP -MF .deps/cpp_message_field.Tpo -c google/protobuf/compiler/cpp/cpp_message_field.cc  -fPIC -DPIC -o cpp_message_field.o
mv -f .deps/cpp_helpers.Tpo .deps/cpp_helpers.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_primitive_field.lo -MD -MP -MF .deps/cpp_primitive_field.Tpo -c -o cpp_primitive_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_primitive_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_primitive_field.cc
mv -f .deps/cpp_message_field.Tpo .deps/cpp_message_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_service.lo -MD -MP -MF .deps/cpp_service.Tpo -c -o cpp_service.lo `test -f 'google/protobuf/compiler/cpp/cpp_service.cc' || echo './'`google/protobuf/compiler/cpp/cpp_service.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_primitive_field.lo -MD -MP -MF .deps/cpp_primitive_field.Tpo -c google/protobuf/compiler/cpp/cpp_primitive_field.cc  -fPIC -DPIC -o cpp_primitive_field.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_service.lo -MD -MP -MF .deps/cpp_service.Tpo -c google/protobuf/compiler/cpp/cpp_service.cc  -fPIC -DPIC -o cpp_service.o
mv -f .deps/cpp_message.Tpo .deps/cpp_message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_string_field.lo -MD -MP -MF .deps/cpp_string_field.Tpo -c -o cpp_string_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_string_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_string_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_string_field.lo -MD -MP -MF .deps/cpp_string_field.Tpo -c google/protobuf/compiler/cpp/cpp_string_field.cc  -fPIC -DPIC -o cpp_string_field.o
mv -f .deps/cpp_service.Tpo .deps/cpp_service.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum.lo -MD -MP -MF .deps/java_enum.Tpo -c -o java_enum.lo `test -f 'google/protobuf/compiler/java/java_enum.cc' || echo './'`google/protobuf/compiler/java/java_enum.cc
mv -f .deps/cpp_primitive_field.Tpo .deps/cpp_primitive_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum_field.lo -MD -MP -MF .deps/java_enum_field.Tpo -c -o java_enum_field.lo `test -f 'google/protobuf/compiler/java/java_enum_field.cc' || echo './'`google/protobuf/compiler/java/java_enum_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum.lo -MD -MP -MF .deps/java_enum.Tpo -c google/protobuf/compiler/java/java_enum.cc  -fPIC -DPIC -o java_enum.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum_field.lo -MD -MP -MF .deps/java_enum_field.Tpo -c google/protobuf/compiler/java/java_enum_field.cc  -fPIC -DPIC -o java_enum_field.o
mv -f .deps/cpp_string_field.Tpo .deps/cpp_string_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_extension.lo -MD -MP -MF .deps/java_extension.Tpo -c -o java_extension.lo `test -f 'google/protobuf/compiler/java/java_extension.cc' || echo './'`google/protobuf/compiler/java/java_extension.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_extension.lo -MD -MP -MF .deps/java_extension.Tpo -c google/protobuf/compiler/java/java_extension.cc  -fPIC -DPIC -o java_extension.o
mv -f .deps/java_enum.Tpo .deps/java_enum.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_field.lo -MD -MP -MF .deps/java_field.Tpo -c -o java_field.lo `test -f 'google/protobuf/compiler/java/java_field.cc' || echo './'`google/protobuf/compiler/java/java_field.cc
mv -f .deps/java_enum_field.Tpo .deps/java_enum_field.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_field.lo -MD -MP -MF .deps/java_field.Tpo -c google/protobuf/compiler/java/java_field.cc  -fPIC -DPIC -o java_field.o
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_file.lo -MD -MP -MF .deps/java_file.Tpo -c -o java_file.lo `test -f 'google/protobuf/compiler/java/java_file.cc' || echo './'`google/protobuf/compiler/java/java_file.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_file.lo -MD -MP -MF .deps/java_file.Tpo -c google/protobuf/compiler/java/java_file.cc  -fPIC -DPIC -o java_file.o
mv -f .deps/java_extension.Tpo .deps/java_extension.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_generator.lo -MD -MP -MF .deps/java_generator.Tpo -c -o java_generator.lo `test -f 'google/protobuf/compiler/java/java_generator.cc' || echo './'`google/protobuf/compiler/java/java_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_generator.lo -MD -MP -MF .deps/java_generator.Tpo -c google/protobuf/compiler/java/java_generator.cc  -fPIC -DPIC -o java_generator.o
mv -f .deps/java_field.Tpo .deps/java_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_helpers.lo -MD -MP -MF .deps/java_helpers.Tpo -c -o java_helpers.lo `test -f 'google/protobuf/compiler/java/java_helpers.cc' || echo './'`google/protobuf/compiler/java/java_helpers.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_helpers.lo -MD -MP -MF .deps/java_helpers.Tpo -c google/protobuf/compiler/java/java_helpers.cc  -fPIC -DPIC -o java_helpers.o
mv -f .deps/java_file.Tpo .deps/java_file.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message.lo -MD -MP -MF .deps/java_message.Tpo -c -o java_message.lo `test -f 'google/protobuf/compiler/java/java_message.cc' || echo './'`google/protobuf/compiler/java/java_message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message.lo -MD -MP -MF .deps/java_message.Tpo -c google/protobuf/compiler/java/java_message.cc  -fPIC -DPIC -o java_message.o
mv -f .deps/java_generator.Tpo .deps/java_generator.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message_field.lo -MD -MP -MF .deps/java_message_field.Tpo -c -o java_message_field.lo `test -f 'google/protobuf/compiler/java/java_message_field.cc' || echo './'`google/protobuf/compiler/java/java_message_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message_field.lo -MD -MP -MF .deps/java_message_field.Tpo -c google/protobuf/compiler/java/java_message_field.cc  -fPIC -DPIC -o java_message_field.o
mv -f .deps/java_helpers.Tpo .deps/java_helpers.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_primitive_field.lo -MD -MP -MF .deps/java_primitive_field.Tpo -c -o java_primitive_field.lo `test -f 'google/protobuf/compiler/java/java_primitive_field.cc' || echo './'`google/protobuf/compiler/java/java_primitive_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_primitive_field.lo -MD -MP -MF .deps/java_primitive_field.Tpo -c google/protobuf/compiler/java/java_primitive_field.cc  -fPIC -DPIC -o java_primitive_field.o
mv -f .deps/java_message.Tpo .deps/java_message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_service.lo -MD -MP -MF .deps/java_service.Tpo -c -o java_service.lo `test -f 'google/protobuf/compiler/java/java_service.cc' || echo './'`google/protobuf/compiler/java/java_service.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_service.lo -MD -MP -MF .deps/java_service.Tpo -c google/protobuf/compiler/java/java_service.cc  -fPIC -DPIC -o java_service.o
mv -f .deps/java_message_field.Tpo .deps/java_message_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_string_field.lo -MD -MP -MF .deps/java_string_field.Tpo -c -o java_string_field.lo `test -f 'google/protobuf/compiler/java/java_string_field.cc' || echo './'`google/protobuf/compiler/java/java_string_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_string_field.lo -MD -MP -MF .deps/java_string_field.Tpo -c google/protobuf/compiler/java/java_string_field.cc  -fPIC -DPIC -o java_string_field.o
mv -f .deps/java_primitive_field.Tpo .deps/java_primitive_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_doc_comment.lo -MD -MP -MF .deps/java_doc_comment.Tpo -c -o java_doc_comment.lo `test -f 'google/protobuf/compiler/java/java_doc_comment.cc' || echo './'`google/protobuf/compiler/java/java_doc_comment.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_doc_comment.lo -MD -MP -MF .deps/java_doc_comment.Tpo -c google/protobuf/compiler/java/java_doc_comment.cc  -fPIC -DPIC -o java_doc_comment.o
mv -f .deps/java_doc_comment.Tpo .deps/java_doc_comment.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT python_generator.lo -MD -MP -MF .deps/python_generator.Tpo -c -o python_generator.lo `test -f 'google/protobuf/compiler/python/python_generator.cc' || echo './'`google/protobuf/compiler/python/python_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT python_generator.lo -MD -MP -MF .deps/python_generator.Tpo -c google/protobuf/compiler/python/python_generator.cc  -fPIC -DPIC -o python_generator.o
mv -f .deps/java_service.Tpo .deps/java_service.Plo
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -version-info 8:0:0 -export-dynamic -no-undefined  -o libprotobuf.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib atomicops_internals_x86_gcc.lo atomicops_internals_x86_msvc.lo common.lo once.lo stringprintf.lo extension_set.lo generated_message_util.lo message_lite.lo repeated_field.lo wire_format_lite.lo coded_stream.lo zero_copy_stream.lo zero_copy_stream_impl_lite.lo strutil.lo substitute.lo structurally_valid.lo descriptor.lo descriptor.pb.lo descriptor_database.lo dynamic_message.lo extension_set_heavy.lo generated_message_reflection.lo message.lo reflection_ops.lo service.lo text_format.lo unknown_field_set.lo wire_format.lo gzip_stream.lo printer.lo tokenizer.lo zero_copy_stream_impl.lo importer.lo parser.lo -lpthread -lz 
libtool: link: ar cru .libs/libprotobuf.a  atomicops_internals_x86_gcc.o atomicops_internals_x86_msvc.o common.o once.o stringprintf.o extension_set.o generated_message_util.o message_lite.o repeated_field.o wire_format_lite.o coded_stream.o zero_copy_stream.o zero_copy_stream_impl_lite.o strutil.o substitute.o structurally_valid.o descriptor.o descriptor.pb.o descriptor_database.o dynamic_message.o extension_set_heavy.o generated_message_reflection.o message.o reflection_ops.o service.o text_format.o unknown_field_set.o wire_format.o gzip_stream.o printer.o tokenizer.o zero_copy_stream_impl.o importer.o parser.o
libtool: link: ranlib .libs/libprotobuf.a
libtool: link: ( cd ".libs" && rm -f "libprotobuf.la" && ln -s "../libprotobuf.la" "libprotobuf.la" )
mv -f .deps/java_string_field.Tpo .deps/java_string_field.Plo
mv -f .deps/python_generator.Tpo .deps/python_generator.Plo
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -version-info 8:0:0 -export-dynamic -no-undefined  -o libprotoc.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib code_generator.lo command_line_interface.lo plugin.lo plugin.pb.lo subprocess.lo zip_writer.lo cpp_enum.lo cpp_enum_field.lo cpp_extension.lo cpp_field.lo cpp_file.lo cpp_generator.lo cpp_helpers.lo cpp_message.lo cpp_message_field.lo cpp_primitive_field.lo cpp_service.lo cpp_string_field.lo java_enum.lo java_enum_field.lo java_extension.lo java_field.lo java_file.lo java_generator.lo java_helpers.lo java_message.lo java_message_field.lo java_primitive_field.lo java_service.lo java_string_field.lo java_doc_comment.lo python_generator.lo -lpthread libprotobuf.la -lz 
libtool: link: ar cru .libs/libprotoc.a  code_generator.o command_line_interface.o plugin.o plugin.pb.o subprocess.o zip_writer.o cpp_enum.o cpp_enum_field.o cpp_extension.o cpp_field.o cpp_file.o cpp_generator.o cpp_helpers.o cpp_message.o cpp_message_field.o cpp_primitive_field.o cpp_service.o cpp_string_field.o java_enum.o java_enum_field.o java_extension.o java_field.o java_file.o java_generator.o java_helpers.o java_message.o java_message_field.o java_primitive_field.o java_service.o java_string_field.o java_doc_comment.o python_generator.o
libtool: link: ranlib .libs/libprotoc.a
libtool: link: ( cd ".libs" && rm -f "libprotoc.la" && ln -s "../libprotoc.la" "libprotoc.la" )
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o protoc main.o -lpthread libprotobuf.la libprotoc.la -lz 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o protoc main.o  ./.libs/libprotobuf.a ./.libs/libprotoc.a /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/.libs/libprotobuf.a -lpthread -lz -pthread
oldpwd=`pwd` && ( cd . && $oldpwd/protoc -I. --cpp_out=$oldpwd google/protobuf/unittest.proto google/protobuf/unittest_empty.proto google/protobuf/unittest_import.proto google/protobuf/unittest_import_public.proto google/protobuf/unittest_mset.proto google/protobuf/unittest_optimize_for.proto google/protobuf/unittest_embed_optimize_for.proto google/protobuf/unittest_custom_options.proto google/protobuf/unittest_lite.proto google/protobuf/unittest_import_lite.proto google/protobuf/unittest_import_public_lite.proto google/protobuf/unittest_lite_imports_nonlite.proto google/protobuf/unittest_no_generic_services.proto google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto )
touch unittest_proto_middleman
make  all-am
make[11]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -version-info 8:0:0 -export-dynamic -no-undefined  -o libprotobuf-lite.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib atomicops_internals_x86_gcc.lo atomicops_internals_x86_msvc.lo common.lo once.lo stringprintf.lo extension_set.lo generated_message_util.lo message_lite.lo repeated_field.lo wire_format_lite.lo coded_stream.lo zero_copy_stream.lo zero_copy_stream_impl_lite.lo -lpthread -lz 
libtool: link: ar cru .libs/libprotobuf-lite.a  atomicops_internals_x86_gcc.o atomicops_internals_x86_msvc.o common.o once.o stringprintf.o extension_set.o generated_message_util.o message_lite.o repeated_field.o wire_format_lite.o coded_stream.o zero_copy_stream.o zero_copy_stream_impl_lite.o
libtool: link: ranlib .libs/libprotobuf-lite.a
libtool: link: ( cd ".libs" && rm -f "libprotobuf-lite.la" && ln -s "../libprotobuf-lite.la" "libprotobuf-lite.la" )
make[11]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
make[10]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
make[9]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
touch protobuf-2.5.0-build-stamp
make[7]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
make[6]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
Making all in .
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-clock.lo -MD -MP -MF .deps/libprocess_la-clock.Tpo -c -o libprocess_la-clock.lo `test -f 'src/clock.cpp' || echo '../../../3rdparty/libprocess/'`src/clock.cpp
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-help.lo -MD -MP -MF .deps/libprocess_la-help.Tpo -c -o libprocess_la-help.lo `test -f 'src/help.cpp' || echo '../../../3rdparty/libprocess/'`src/help.cpp
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-http.lo -MD -MP -MF .deps/libprocess_la-http.Tpo -c -o libprocess_la-http.lo `test -f 'src/http.cpp' || echo '../../../3rdparty/libprocess/'`src/http.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-clock.lo -MD -MP -MF .deps/libprocess_la-clock.Tpo -c ../../../3rdparty/libprocess/src/clock.cpp  -fPIC -DPIC -o libprocess_la-clock.o
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-help.lo -MD -MP -MF .deps/libprocess_la-help.Tpo -c ../../../3rdparty/libprocess/src/help.cpp  -fPIC -DPIC -o libprocess_la-help.o
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-http.lo -MD -MP -MF .deps/libprocess_la-http.Tpo -c ../../../3rdparty/libprocess/src/http.cpp  -fPIC -DPIC -o libprocess_la-http.o
mv -f .deps/libprocess_la-clock.Tpo .deps/libprocess_la-clock.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-io.lo -MD -MP -MF .deps/libprocess_la-io.Tpo -c -o libprocess_la-io.lo `test -f 'src/io.cpp' || echo '../../../3rdparty/libprocess/'`src/io.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-io.lo -MD -MP -MF .deps/libprocess_la-io.Tpo -c ../../../3rdparty/libprocess/src/io.cpp  -fPIC -DPIC -o libprocess_la-io.o
mv -f .deps/libprocess_la-help.Tpo .deps/libprocess_la-help.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-latch.lo -MD -MP -MF .deps/libprocess_la-latch.Tpo -c -o libprocess_la-latch.lo `test -f 'src/latch.cpp' || echo '../../../3rdparty/libprocess/'`src/latch.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-latch.lo -MD -MP -MF .deps/libprocess_la-latch.Tpo -c ../../../3rdparty/libprocess/src/latch.cpp  -fPIC -DPIC -o libprocess_la-latch.o
mv -f .deps/libprocess_la-http.Tpo .deps/libprocess_la-http.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-metrics.lo -MD -MP -MF .deps/libprocess_la-metrics.Tpo -c -o libprocess_la-metrics.lo `test -f 'src/metrics/metrics.cpp' || echo '../../../3rdparty/libprocess/'`src/metrics/metrics.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-metrics.lo -MD -MP -MF .deps/libprocess_la-metrics.Tpo -c ../../../3rdparty/libprocess/src/metrics/metrics.cpp  -fPIC -DPIC -o libprocess_la-metrics.o
mv -f .deps/libprocess_la-latch.Tpo .deps/libprocess_la-latch.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-pid.lo -MD -MP -MF .deps/libprocess_la-pid.Tpo -c -o libprocess_la-pid.lo `test -f 'src/pid.cpp' || echo '../../../3rdparty/libprocess/'`src/pid.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-pid.lo -MD -MP -MF .deps/libprocess_la-pid.Tpo -c ../../../3rdparty/libprocess/src/pid.cpp  -fPIC -DPIC -o libprocess_la-pid.o
mv -f .deps/libprocess_la-io.Tpo .deps/libprocess_la-io.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-poll_socket.lo -MD -MP -MF .deps/libprocess_la-poll_socket.Tpo -c -o libprocess_la-poll_socket.lo `test -f 'src/poll_socket.cpp' || echo '../../../3rdparty/libprocess/'`src/poll_socket.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-poll_socket.lo -MD -MP -MF .deps/libprocess_la-poll_socket.Tpo -c ../../../3rdparty/libprocess/src/poll_socket.cpp  -fPIC -DPIC -o libprocess_la-poll_socket.o
mv -f .deps/libprocess_la-pid.Tpo .deps/libprocess_la-pid.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-process.lo -MD -MP -MF .deps/libprocess_la-process.Tpo -c -o libprocess_la-process.lo `test -f 'src/process.cpp' || echo '../../../3rdparty/libprocess/'`src/process.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-process.lo -MD -MP -MF .deps/libprocess_la-process.Tpo -c ../../../3rdparty/libprocess/src/process.cpp  -fPIC -DPIC -o libprocess_la-process.o
mv -f .deps/libprocess_la-poll_socket.Tpo .deps/libprocess_la-poll_socket.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c -o libprocess_la-reap.lo `test -f 'src/reap.cpp' || echo '../../../3rdparty/libprocess/'`src/reap.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c ../../../3rdparty/libprocess/src/reap.cpp  -fPIC -DPIC -o libprocess_la-reap.o
mv -f .deps/libprocess_la-metrics.Tpo .deps/libprocess_la-metrics.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-socket.lo -MD -MP -MF .deps/libprocess_la-socket.Tpo -c -o libprocess_la-socket.lo `test -f 'src/socket.cpp' || echo '../../../3rdparty/libprocess/'`src/socket.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-socket.lo -MD -MP -MF .deps/libprocess_la-socket.Tpo -c ../../../3rdparty/libprocess/src/socket.cpp  -fPIC -DPIC -o libprocess_la-socket.o
mv -f .deps/libprocess_la-reap.Tpo .deps/libprocess_la-reap.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-subprocess.lo -MD -MP -MF .deps/libprocess_la-subprocess.Tpo -c -o libprocess_la-subprocess.lo `test -f 'src/subprocess.cpp' || echo '../../../3rdparty/libprocess/'`src/subprocess.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-subprocess.lo -MD -MP -MF .deps/libprocess_la-subprocess.Tpo -c ../../../3rdparty/libprocess/src/subprocess.cpp  -fPIC -DPIC -o libprocess_la-subprocess.o
mv -f .deps/libprocess_la-socket.Tpo .deps/libprocess_la-socket.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-timeseries.lo -MD -MP -MF .deps/libprocess_la-timeseries.Tpo -c -o libprocess_la-timeseries.lo `test -f 'src/timeseries.cpp' || echo '../../../3rdparty/libprocess/'`src/timeseries.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-timeseries.lo -MD -MP -MF .deps/libprocess_la-timeseries.Tpo -c ../../../3rdparty/libprocess/src/timeseries.cpp  -fPIC -DPIC -o libprocess_la-timeseries.o
mv -f .deps/libprocess_la-timeseries.Tpo .deps/libprocess_la-timeseries.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev.lo -MD -MP -MF .deps/libprocess_la-libev.Tpo -c -o libprocess_la-libev.lo `test -f 'src/libev.cpp' || echo '../../../3rdparty/libprocess/'`src/libev.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev.lo -MD -MP -MF .deps/libprocess_la-libev.Tpo -c ../../../3rdparty/libprocess/src/libev.cpp  -fPIC -DPIC -o libprocess_la-libev.o
mv -f .deps/libprocess_la-subprocess.Tpo .deps/libprocess_la-subprocess.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev_poll.lo -MD -MP -MF .deps/libprocess_la-libev_poll.Tpo -c -o libprocess_la-libev_poll.lo `test -f 'src/libev_poll.cpp' || echo '../../../3rdparty/libprocess/'`src/libev_poll.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev_poll.lo -MD -MP -MF .deps/libprocess_la-libev_poll.Tpo -c ../../../3rdparty/libprocess/src/libev_poll.cpp  -fPIC -DPIC -o libprocess_la-libev_poll.o
mv -f .deps/libprocess_la-libev.Tpo .deps/libprocess_la-libev.Plo
mv -f .deps/libprocess_la-process.Tpo .deps/libprocess_la-process.Plo
mv -f .deps/libprocess_la-libev_poll.Tpo .deps/libprocess_la-libev_poll.Plo
/bin/bash ./libtool  --tag=CXX   --mode=link g++  -g1 -O0 -Wno-unused-local-typedefs -std=c++11   -o libprocess.la  libprocess_la-clock.lo libprocess_la-help.lo libprocess_la-http.lo libprocess_la-io.lo libprocess_la-latch.lo libprocess_la-metrics.lo libprocess_la-pid.lo libprocess_la-poll_socket.lo libprocess_la-process.lo libprocess_la-reap.lo libprocess_la-socket.lo libprocess_la-subprocess.lo libprocess_la-timeseries.lo  libprocess_la-libev.lo libprocess_la-libev_poll.lo 3rdparty/glog-0.3.3/libglog.la 3rdparty/libry_http_parser.la 3rdparty/libev-4.15/libev.la  -ldl -lz -lcurl -lsvn_delta-1 -lsvn_subr-1 -lapr-1  -lrt
libtool: link: (cd .libs/libprocess.lax/libry_http_parser.a && ar x "/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/.libs/libry_http_parser.a")
libtool: link: ar cru .libs/libprocess.a  libprocess_la-clock.o libprocess_la-help.o libprocess_la-http.o libprocess_la-io.o libprocess_la-latch.o libprocess_la-metrics.o libprocess_la-pid.o libprocess_la-poll_socket.o libprocess_la-process.o libprocess_la-reap.o libprocess_la-socket.o libprocess_la-subprocess.o libprocess_la-timeseries.o libprocess_la-libev.o libprocess_la-libev_poll.o  .libs/libprocess.lax/libry_http_parser.a/libry_http_parser_la-http_parser.o 
libtool: link: ranlib .libs/libprocess.a
libtool: link: rm -fr .libs/libprocess.lax
libtool: link: ( cd ".libs" && rm -f "libprocess.la" && ln -s "../libprocess.la" "libprocess.la" )
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
Making all in include
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/include'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/include'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
gzip -d -c ../../3rdparty/distribute-0.6.26.tar.gz | tar xf -
gzip -d -c ../../3rdparty/pip-1.5.6.tar.gz | tar xf -
gzip -d -c ../../3rdparty/wheel-0.24.0.tar.gz | tar xf -
test ! -e ../../3rdparty/wheel-0.24.0.patch || patch -d wheel-0.24.0 -p1 <../../3rdparty/wheel-0.24.0.patch
touch wheel-0.24.0-stamp
gzip -d -c ../../3rdparty/leveldb.tar.gz | tar xf -
test ! -e ../../3rdparty/distribute-0.6.26.patch || patch -d distribute-0.6.26 -p1 <../../3rdparty/distribute-0.6.26.patch
touch distribute-0.6.26-stamp
test ! -e ../../3rdparty/leveldb.patch || patch -d leveldb -p1 <../../3rdparty/leveldb.patch
gzip -d -c ../../3rdparty/zookeeper-3.4.5.tar.gz | tar xf -
patching file build_detect_platform
touch leveldb-stamp
cd leveldb && \
          make  CC="gcc" CXX="g++" OPT="-g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC"
test ! -e ../../3rdparty/pip-1.5.6.patch || patch -d pip-1.5.6 -p1 <../../3rdparty/pip-1.5.6.patch
touch pip-1.5.6-stamp
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/leveldb'
g++ -pthread -shared -Wl,-soname -Wl,/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/leveldb/libleveldb.so.1 -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -fPIC db/builder.cc db/c.cc db/dbformat.cc db/db_impl.cc db/db_iter.cc db/filename.cc db/log_reader.cc db/log_writer.cc db/memtable.cc db/repair.cc db/table_cache.cc db/version_edit.cc db/version_set.cc db/write_batch.cc table/block_builder.cc table/block.cc table/filter_block.cc table/format.cc table/iterator.cc table/merger.cc table/table_builder.cc table/table.cc table/two_level_iterator.cc util/arena.cc util/bloom.cc util/cache.cc util/coding.cc util/comparator.cc util/crc32c.cc util/env.cc util/env_posix.cc util/filter_policy.cc util/hash.cc util/histogram.cc util/logging.cc util/options.cc util/status.cc  port/port_posix.cc -o libleveldb.so.1.4
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/builder.cc -o db/builder.o
test ! -e ../../3rdparty/zookeeper-3.4.5.patch || patch -d zookeeper-3.4.5 -p1 <../../3rdparty/zookeeper-3.4.5.patch
patching file src/c/include/recordio.h
patching file src/c/src/recordio.c
patching file src/c/src/zookeeper.c
patching file src/c/tests/ZKMocks.cc
touch zookeeper-3.4.5-stamp
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/c.cc -o db/c.o
cd zookeeper-3.4.5/src/c && ./configure '--disable-python-dependency-install' '--srcdir=..' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
	  make 
checking for doxygen... no
checking for perl... /usr/bin/perl
checking for dot... no
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/dbformat.cc -o db/dbformat.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/db_impl.cc -o db/db_impl.o
yes
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking for cppunit-config... /usr/bin/cppunit-config
checking for Cppunit - version >= 1.10.2... 1.13.1
checking for generated/zookeeper.jute.c... yes
checking for generated/zookeeper.jute.h... yes
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking whether gcc and cc understand -c and -o together... yes
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln -s works... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for a sed that does not truncate output... /bin/sed
checking for egrep... grep -E
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for /usr/bin/ld option to reload object files... -r
checking for BSD-compatible nm... /usr/bin/nm -B
checking how to recognise dependent libraries... pass_all
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/db_iter.cc -o db/db_iter.o
yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/filename.cc -o db/filename.o
yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking dlfcn.h usability... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/log_reader.cc -o db/log_reader.o
yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking how to run the C++ preprocessor... g++ -E
checking for g77... no
checking for f77... no
checking for xlf... no
checking for frt... no
checking for pgf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for f90... no
checking for xlf90... no
checking for pgf90... no
checking for epcf90... no
checking for f95... no
checking for fort... no
checking for xlf95... no
checking for ifc... no
checking for efc... no
checking for pgf95... no
checking for lf95... no
checking for gfortran... no
checking whether we are using the GNU Fortran 77 compiler... no
checking whether  accepts -g... no
checking the maximum length of command line arguments... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/log_writer.cc -o db/log_writer.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/memtable.cc -o db/memtable.o
32768
checking command to parse /usr/bin/nm -B output from gcc object... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/repair.cc -o db/repair.o
ok
checking for objdir... .libs
checking for ar... ar
checking for ranlib... ranlib
checking for strip... strip
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC
checking if gcc PIC flag -fPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
configure: creating libtool
appending configuration tag "CXX" to libtool
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC
checking if g++ PIC flag -fPIC works... yes
checking if g++ static flag -static works... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/table_cache.cc -o db/table_cache.o
yes
checking if g++ supports -c -o file.o... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
appending configuration tag "F77" to libtool
checking for pthread_mutex_lock in -lpthread... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/version_edit.cc -o db/version_edit.o
yes
configure: building with SyncAPI support
checking for ANSI C header files... (cached) yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/time.h usability... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/version_set.cc -o db/version_set.o
yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking sys/utsname.h usability... yes
checking sys/utsname.h presence... yes
checking for sys/utsname.h... yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether time.h and sys/time.h may both be included... yes
checking for nfds_t... yes
checking whether to enable ipv6... yes
checking for getcwd... yes
checking for gethostbyname... yes
checking for gethostname... yes
checking for getlogin... yes
checking for getpwuid_r... yes
checking for gettimeofday... yes
checking for getuid... yes
checking for memmove... yes
checking for memset... yes
checking for poll... yes
checking for socket... yes
checking for strchr... yes
checking for strdup... yes
checking for strerror... yes
checking for strtol... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/write_batch.cc -o db/write_batch.o
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
make  all-am
make[6]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zookeeper.lo -MD -MP -MF ".deps/zookeeper.Tpo" -c -o zookeeper.lo `test -f 'src/zookeeper.c' || echo './'`src/zookeeper.c; \
	then mv -f ".deps/zookeeper.Tpo" ".deps/zookeeper.Plo"; else rm -f ".deps/zookeeper.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zookeeper.lo -MD -MP -MF .deps/zookeeper.Tpo -c src/zookeeper.c  -fPIC -DPIC -o zookeeper.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT recordio.lo -MD -MP -MF ".deps/recordio.Tpo" -c -o recordio.lo `test -f 'src/recordio.c' || echo './'`src/recordio.c; \
	then mv -f ".deps/recordio.Tpo" ".deps/recordio.Plo"; else rm -f ".deps/recordio.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT recordio.lo -MD -MP -MF .deps/recordio.Tpo -c src/recordio.c  -fPIC -DPIC -o recordio.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zookeeper.jute.lo -MD -MP -MF ".deps/zookeeper.jute.Tpo" -c -o zookeeper.jute.lo `test -f 'generated/zookeeper.jute.c' || echo './'`generated/zookeeper.jute.c; \
	then mv -f ".deps/zookeeper.jute.Tpo" ".deps/zookeeper.jute.Plo"; else rm -f ".deps/zookeeper.jute.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zookeeper.jute.lo -MD -MP -MF .deps/zookeeper.jute.Tpo -c generated/zookeeper.jute.c  -fPIC -DPIC -o zookeeper.jute.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/block_builder.cc -o table/block_builder.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zk_log.lo -MD -MP -MF ".deps/zk_log.Tpo" -c -o zk_log.lo `test -f 'src/zk_log.c' || echo './'`src/zk_log.c; \
	then mv -f ".deps/zk_log.Tpo" ".deps/zk_log.Plo"; else rm -f ".deps/zk_log.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zk_log.lo -MD -MP -MF .deps/zk_log.Tpo -c src/zk_log.c  -fPIC -DPIC -o zk_log.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zk_hashtable.lo -MD -MP -MF ".deps/zk_hashtable.Tpo" -c -o zk_hashtable.lo `test -f 'src/zk_hashtable.c' || echo './'`src/zk_hashtable.c; \
	then mv -f ".deps/zk_hashtable.Tpo" ".deps/zk_hashtable.Plo"; else rm -f ".deps/zk_hashtable.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zk_hashtable.lo -MD -MP -MF .deps/zk_hashtable.Tpo -c src/zk_hashtable.c  -fPIC -DPIC -o zk_hashtable.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT st_adaptor.lo -MD -MP -MF ".deps/st_adaptor.Tpo" -c -o st_adaptor.lo `test -f 'src/st_adaptor.c' || echo './'`src/st_adaptor.c; \
	then mv -f ".deps/st_adaptor.Tpo" ".deps/st_adaptor.Plo"; else rm -f ".deps/st_adaptor.Tpo"; exit 1; fi
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/block.cc -o table/block.o
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT st_adaptor.lo -MD -MP -MF .deps/st_adaptor.Tpo -c src/st_adaptor.c  -fPIC -DPIC -o st_adaptor.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT hashtable_itr.lo -MD -MP -MF ".deps/hashtable_itr.Tpo" -c -o hashtable_itr.lo `test -f 'src/hashtable/hashtable_itr.c' || echo './'`src/hashtable/hashtable_itr.c; \
	then mv -f ".deps/hashtable_itr.Tpo" ".deps/hashtable_itr.Plo"; else rm -f ".deps/hashtable_itr.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT hashtable_itr.lo -MD -MP -MF .deps/hashtable_itr.Tpo -c src/hashtable/hashtable_itr.c  -fPIC -DPIC -o hashtable_itr.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT hashtable.lo -MD -MP -MF ".deps/hashtable.Tpo" -c -o hashtable.lo `test -f 'src/hashtable/hashtable.c' || echo './'`src/hashtable/hashtable.c; \
	then mv -f ".deps/hashtable.Tpo" ".deps/hashtable.Plo"; else rm -f ".deps/hashtable.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT hashtable.lo -MD -MP -MF .deps/hashtable.Tpo -c src/hashtable/hashtable.c  -fPIC -DPIC -o hashtable.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.lo -MD -MP -MF ".deps/libzkmt_la-zookeeper.Tpo" -c -o libzkmt_la-zookeeper.lo `test -f 'src/zookeeper.c' || echo './'`src/zookeeper.c; \
	then mv -f ".deps/libzkmt_la-zookeeper.Tpo" ".deps/libzkmt_la-zookeeper.Plo"; else rm -f ".deps/libzkmt_la-zookeeper.Tpo"; exit 1; fi
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-recordio.lo -MD -MP -MF ".deps/libzkmt_la-recordio.Tpo" -c -o libzkmt_la-recordio.lo `test -f 'src/recordio.c' || echo './'`src/recordio.c; \
	then mv -f ".deps/libzkmt_la-recordio.Tpo" ".deps/libzkmt_la-recordio.Plo"; else rm -f ".deps/libzkmt_la-recordio.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.lo -MD -MP -MF .deps/libzkmt_la-zookeeper.Tpo -c src/zookeeper.c  -fPIC -DPIC -o libzkmt_la-zookeeper.o
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-recordio.lo -MD -MP -MF .deps/libzkmt_la-recordio.Tpo -c src/recordio.c  -fPIC -DPIC -o libzkmt_la-recordio.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.jute.lo -MD -MP -MF ".deps/libzkmt_la-zookeeper.jute.Tpo" -c -o libzkmt_la-zookeeper.jute.lo `test -f 'generated/zookeeper.jute.c' || echo './'`generated/zookeeper.jute.c; \
	then mv -f ".deps/libzkmt_la-zookeeper.jute.Tpo" ".deps/libzkmt_la-zookeeper.jute.Plo"; else rm -f ".deps/libzkmt_la-zookeeper.jute.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.jute.lo -MD -MP -MF .deps/libzkmt_la-zookeeper.jute.Tpo -c generated/zookeeper.jute.c  -fPIC -DPIC -o libzkmt_la-zookeeper.jute.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/filter_block.cc -o table/filter_block.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_log.lo -MD -MP -MF ".deps/libzkmt_la-zk_log.Tpo" -c -o libzkmt_la-zk_log.lo `test -f 'src/zk_log.c' || echo './'`src/zk_log.c; \
	then mv -f ".deps/libzkmt_la-zk_log.Tpo" ".deps/libzkmt_la-zk_log.Plo"; else rm -f ".deps/libzkmt_la-zk_log.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_log.lo -MD -MP -MF .deps/libzkmt_la-zk_log.Tpo -c src/zk_log.c  -fPIC -DPIC -o libzkmt_la-zk_log.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_hashtable.lo -MD -MP -MF ".deps/libzkmt_la-zk_hashtable.Tpo" -c -o libzkmt_la-zk_hashtable.lo `test -f 'src/zk_hashtable.c' || echo './'`src/zk_hashtable.c; \
	then mv -f ".deps/libzkmt_la-zk_hashtable.Tpo" ".deps/libzkmt_la-zk_hashtable.Plo"; else rm -f ".deps/libzkmt_la-zk_hashtable.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_hashtable.lo -MD -MP -MF .deps/libzkmt_la-zk_hashtable.Tpo -c src/zk_hashtable.c  -fPIC -DPIC -o libzkmt_la-zk_hashtable.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/format.cc -o table/format.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-mt_adaptor.lo -MD -MP -MF ".deps/libzkmt_la-mt_adaptor.Tpo" -c -o libzkmt_la-mt_adaptor.lo `test -f 'src/mt_adaptor.c' || echo './'`src/mt_adaptor.c; \
	then mv -f ".deps/libzkmt_la-mt_adaptor.Tpo" ".deps/libzkmt_la-mt_adaptor.Plo"; else rm -f ".deps/libzkmt_la-mt_adaptor.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-mt_adaptor.lo -MD -MP -MF .deps/libzkmt_la-mt_adaptor.Tpo -c src/mt_adaptor.c  -fPIC -DPIC -o libzkmt_la-mt_adaptor.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/iterator.cc -o table/iterator.o
if gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT cli.o -MD -MP -MF ".deps/cli.Tpo" -c -o cli.o `test -f 'src/cli.c' || echo './'`src/cli.c; \
	then mv -f ".deps/cli.Tpo" ".deps/cli.Po"; else rm -f ".deps/cli.Tpo"; exit 1; fi
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/merger.cc -o table/merger.o
if gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT cli_mt-cli.o -MD -MP -MF ".deps/cli_mt-cli.Tpo" -c -o cli_mt-cli.o `test -f 'src/cli.c' || echo './'`src/cli.c; \
	then mv -f ".deps/cli_mt-cli.Tpo" ".deps/cli_mt-cli.Po"; else rm -f ".deps/cli_mt-cli.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT load_gen-load_gen.o -MD -MP -MF ".deps/load_gen-load_gen.Tpo" -c -o load_gen-load_gen.o `test -f 'src/load_gen.c' || echo './'`src/load_gen.c; \
	then mv -f ".deps/load_gen-load_gen.Tpo" ".deps/load_gen-load_gen.Po"; else rm -f ".deps/load_gen-load_gen.Tpo"; exit 1; fi
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzkst.la   zookeeper.lo recordio.lo zookeeper.jute.lo zk_log.lo zk_hashtable.lo st_adaptor.lo -lm 
mkdir .libs
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libhashtable.la   hashtable_itr.lo hashtable.lo  
ar cru .libs/libzkst.a  zookeeper.o recordio.o zookeeper.jute.o zk_log.o zk_hashtable.o st_adaptor.o
ranlib .libs/libzkst.a
creating libzkst.la
(cd .libs && rm -f libzkst.la && ln -s ../libzkst.la libzkst.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzkmt.la   libzkmt_la-zookeeper.lo libzkmt_la-recordio.lo libzkmt_la-zookeeper.jute.lo libzkmt_la-zk_log.lo libzkmt_la-zk_hashtable.lo libzkmt_la-mt_adaptor.lo -lm 
ar cru .libs/libhashtable.a  hashtable_itr.o hashtable.o
ranlib .libs/libhashtable.a
creating libhashtable.la
(cd .libs && rm -f libhashtable.la && ln -s ../libhashtable.la libhashtable.la)
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/table_builder.cc -o table/table_builder.o
ar cru .libs/libzkmt.a  libzkmt_la-zookeeper.o libzkmt_la-recordio.o libzkmt_la-zookeeper.jute.o libzkmt_la-zk_log.o libzkmt_la-zk_hashtable.o libzkmt_la-mt_adaptor.o
ranlib .libs/libzkmt.a
creating libzkmt.la
(cd .libs && rm -f libzkmt.la && ln -s ../libzkmt.la libzkmt.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzookeeper_st.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib -no-undefined -version-info 2 -export-symbols-regex '(zoo_|zookeeper_|zhandle|Z|format_log_message|log_message|logLevel|deallocate_|zerror|is_unrecoverable)'  libzkst.la libhashtable.la 
rm -fr .libs/libzookeeper_st.lax
mkdir .libs/libzookeeper_st.lax
rm -fr .libs/libzookeeper_st.lax/libzkst.a
mkdir .libs/libzookeeper_st.lax/libzkst.a
(cd .libs/libzookeeper_st.lax/libzkst.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libzkst.a)
rm -fr .libs/libzookeeper_st.lax/libhashtable.a
mkdir .libs/libzookeeper_st.lax/libhashtable.a
(cd .libs/libzookeeper_st.lax/libhashtable.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libhashtable.a)
ar cru .libs/libzookeeper_st.a   .libs/libzookeeper_st.lax/libzkst.a/zookeeper.o .libs/libzookeeper_st.lax/libzkst.a/zk_hashtable.o .libs/libzookeeper_st.lax/libzkst.a/st_adaptor.o .libs/libzookeeper_st.lax/libzkst.a/zookeeper.jute.o .libs/libzookeeper_st.lax/libzkst.a/recordio.o .libs/libzookeeper_st.lax/libzkst.a/zk_log.o  .libs/libzookeeper_st.lax/libhashtable.a/hashtable_itr.o .libs/libzookeeper_st.lax/libhashtable.a/hashtable.o 
ranlib .libs/libzookeeper_st.a
rm -fr .libs/libzookeeper_st.lax
creating libzookeeper_st.la
(cd .libs && rm -f libzookeeper_st.la && ln -s ../libzookeeper_st.la libzookeeper_st.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzookeeper_mt.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib -no-undefined -version-info 2 -export-symbols-regex '(zoo_|zookeeper_|zhandle|Z|format_log_message|log_message|logLevel|deallocate_|zerror|is_unrecoverable)'  libzkmt.la libhashtable.la -lpthread 
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/table.cc -o table/table.o
rm -fr .libs/libzookeeper_mt.lax
mkdir .libs/libzookeeper_mt.lax
rm -fr .libs/libzookeeper_mt.lax/libzkmt.a
mkdir .libs/libzookeeper_mt.lax/libzkmt.a
(cd .libs/libzookeeper_mt.lax/libzkmt.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libzkmt.a)
rm -fr .libs/libzookeeper_mt.lax/libhashtable.a
mkdir .libs/libzookeeper_mt.lax/libhashtable.a
(cd .libs/libzookeeper_mt.lax/libhashtable.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libhashtable.a)
ar cru .libs/libzookeeper_mt.a   .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zookeeper.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zk_hashtable.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-mt_adaptor.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zookeeper.jute.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-recordio.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zk_log.o  .libs/libzookeeper_mt.lax/libhashtable.a/hashtable_itr.o .libs/libzookeeper_mt.lax/libhashtable.a/hashtable.o 
ranlib .libs/libzookeeper_mt.a
rm -fr .libs/libzookeeper_mt.lax
creating libzookeeper_mt.la
(cd .libs && rm -f libzookeeper_mt.la && ln -s ../libzookeeper_mt.la libzookeeper_mt.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o cli_st  cli.o libzookeeper_st.la 
gcc -Wall -Werror -g -O2 -D_GNU_SOURCE -o cli_st cli.o  ./.libs/libzookeeper_st.a -lm  
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o cli_mt  cli_mt-cli.o libzookeeper_mt.la 
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/two_level_iterator.cc -o table/two_level_iterator.o
gcc -Wall -Werror -g -O2 -D_GNU_SOURCE -o cli_mt cli_mt-cli.o  ./.libs/libzookeeper_mt.a -lm -lpthread  
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o load_gen  load_gen-load_gen.o libzookeeper_mt.la 
gcc -Wall -Werror -g -O2 -D_GNU_SOURCE -o load_gen load_gen-load_gen.o  ./.libs/libzookeeper_mt.a -lm -lpthread  
make[6]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/arena.cc -o util/arena.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/bloom.cc -o util/bloom.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/cache.cc -o util/cache.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/coding.cc -o util/coding.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/comparator.cc -o util/comparator.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/crc32c.cc -o util/crc32c.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/env.cc -o util/env.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/env_posix.cc -o util/env_posix.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/filter_policy.cc -o util/filter_policy.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/hash.cc -o util/hash.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/histogram.cc -o util/histogram.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/logging.cc -o util/logging.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/options.cc -o util/options.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/status.cc -o util/status.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c port/port_posix.cc -o port/port_posix.o
ln -fs libleveldb.so.1.4 libleveldb.so
ln -fs libleveldb.so.1.4 libleveldb.so.1
rm -f libleveldb.a
ar -rs libleveldb.a db/builder.o db/c.o db/dbformat.o db/db_impl.o db/db_iter.o db/filename.o db/log_reader.o db/log_writer.o db/memtable.o db/repair.o db/table_cache.o db/version_edit.o db/version_set.o db/write_batch.o table/block_builder.o table/block.o table/filter_block.o table/format.o table/iterator.o table/merger.o table/table_builder.o table/table.o table/two_level_iterator.o util/arena.o util/bloom.o util/cache.o util/coding.o util/comparator.o util/crc32c.o util/env.o util/env_posix.o util/filter_policy.o util/hash.o util/histogram.o util/logging.o util/options.o util/status.o port/port_posix.o
ar: creating libleveldb.a
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/leveldb'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
make[3]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
Making all in src
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
/bin/mkdir -p .
/bin/mkdir -p containerizer
/bin/mkdir -p fetcher
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/mesos.proto
/bin/mkdir -p ../include/mesos/containerizer
/bin/mkdir -p ../include/mesos/fetcher
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/containerizer/containerizer.proto
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/fetcher/fetcher.proto
mv ../include/mesos/fetcher/*.pb.cc fetcher
mv ../include/mesos/containerizer/*.pb.cc containerizer
/bin/mkdir -p scheduler
/bin/mkdir -p  java/generated/org/apache/mesos
/bin/mkdir -p ../include/mesos/scheduler
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/scheduler/scheduler.proto
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --java_out=java/generated ../../include/mesos/mesos.proto
mv ../include/mesos/scheduler/*.pb.cc scheduler
/bin/mkdir -p  java/generated/org/apache/mesos/containerizer
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --java_out=java/generated ../../include/mesos/containerizer/containerizer.proto
mv ../include/mesos/*.pb.cc .
/bin/mkdir -p python/interface/src/mesos/interface
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include/mesos				\
		-I../../include -I../../src --python_out=python/interface/src/mesos/interface ../../include/mesos/mesos.proto
/bin/mkdir -p python/interface/src/mesos/interface
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include/mesos/containerizer		\
		-I../../include -I../../src						\
		--python_out=python/interface/src/mesos/interface ../../include/mesos/containerizer/containerizer.proto
/bin/mkdir -p messages
sed -e 's/mesos\.mesos_pb2/mesos_pb2/' <python/interface/src/mesos/interface/containerizer_pb2.py >python/interface/src/mesos/interface/containerizer_pb2.py
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/messages/messages.proto
/bin/mkdir -p messages
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/messages/log.proto
/bin/mkdir -p messages
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/messages/state.proto
/bin/mkdir -p master
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/master/registry.proto
Building mesos-0.22.0.jar ...
[INFO] Scanning for projects...
[INFO] 
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building mesos 0.22.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ mesos ---
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (default) @ mesos ---
[INFO] Source directory: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/generated added.
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ mesos ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ mesos ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/classes
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ mesos ---
[INFO] Compiling 17 source files to /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ mesos ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/src/test/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ mesos ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ mesos ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ mesos ---
[INFO] Building jar: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/mesos-0.22.0.jar
[INFO] 
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ mesos ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (build-and-attach-javadocs) @ mesos ---
[INFO] 
Loading source files for package org.apache.mesos...
Loading source files for package org.apache.mesos.state...
Loading source files for package org.apache.mesos.containerizer...
Constructing Javadoc information...
Standard Doclet version 1.6.0_20
Building tree for all the packages and classes...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Executor.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//ExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Entry.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.OperationFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Position.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Reader.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Writer.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.WriterFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//MesosExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//MesosSchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Scheduler.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//SchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//AbstractState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//InMemoryState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//LevelDBState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//LogState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//State.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//Variable.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//ZooKeeperState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/overview-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-summary.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-tree.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-summary.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-tree.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/constant-values.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/serialized-form.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Scheduler.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/SchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/ExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Executor.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Position.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Entry.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.OperationFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.WriterFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Reader.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Writer.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/MesosExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/MesosSchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/Variable.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/LogState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/ZooKeeperState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/AbstractState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/InMemoryState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/State.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/LevelDBState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-use.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-use.html...
Building index for all the packages and classes...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/overview-tree.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/index-all.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/deprecated-list.html...
Building index for all classes...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/allclasses-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/allclasses-noframe.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/index.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/overview-summary.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/help-doc.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/stylesheet.css...
3 warnings
[WARNING] Javadoc Warnings
[WARNING] javadoc: warning - No source files for package org.apache.mesos.containerizer
[WARNING] javadoc: warning - No source files for package org.apache.mesos.containerizer
[WARNING] /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/../src/java/src/org/apache/mesos/SchedulerDriver.java:161: warning - Tag @link: can't find launchTasks(Collection, Collection, Filters) in org.apache.mesos.SchedulerDriver
[INFO] Building jar: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/mesos-0.22.0-javadoc.jar
[INFO] 
[INFO] >>> maven-source-plugin:2.1.2:jar (attach-sources) @ mesos >>>
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (default) @ mesos ---
[INFO] Source directory: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/generated added.
[INFO] 
[INFO] <<< maven-source-plugin:2.1.2:jar (attach-sources) @ mesos <<<
[INFO] 
[INFO] --- maven-source-plugin:2.1.2:jar (attach-sources) @ mesos ---
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/mesos already added, skipping
[INFO] Building jar: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/mesos-0.22.0-sources.jar
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/mesos already added, skipping
[INFO] 
[INFO] --- maven-shade-plugin:2.2:shade (default) @ mesos ---
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Attaching shaded artifact.
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:copy-dependencies (copy-dependencies) @ mesos ---
[INFO] Copying protobuf-java-2.5.0.jar to /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.124 s
[INFO] Finished at: 2015-02-10T15:03:52+00:00
[INFO] Final Memory: 58M/1301M
[INFO] ------------------------------------------------------------------------
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.MesosSchedulerDriver
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
          org.apache.mesos.MesosExecutorDriver
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.Log
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.AbstractState
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.LevelDBState
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.LogState
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.Variable
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.ZooKeeperState
make  all-am
make[3]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-status_update_manager.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Tpo -c -o slave/libmesos_no_3rdparty_la-status_update_manager.lo `test -f 'slave/status_update_manager.cpp' |
 | echo '../../src/'`slave/status_update_manager.cpp
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-linux_launcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-linux_launcher.lo `test -f 'slave/contain
 erizer/linux_launcher.cpp' || echo '../../src/'`slave/containerizer/linux_launcher.cpp
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libmesos_no_3rdparty_la-mesos.pb.lo -MD -MP -MF .deps/libmesos_no_3rdparty_la-mesos.pb.Tpo -c -o libmesos_no_3rdparty_la-mesos.pb.lo `test -f 'mesos.pb.cc' || echo '../../src/'`mesos.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-status_update_manager.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Tpo -c ../../src/slave/status_update_manager.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-status_update_manager.o
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mlibtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/worksp
 ace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libmesos_no_3rdparty_la-mesos.pb.lo -MD -MP -MF .deps/libmesos_no_3rdparty_la-mesos.pb.Tpo -c mesos.pb.cc  -fPIC -DPIC -o .libs/libmesos_no_3rdparty_la-mesos.pb.o
esos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-linux_launcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Tpo -c ../../src/slave/containerizer/linux_launcher.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-linux_launcher.o
mv -f .deps/libmesos_no_3rdparty_la-mesos.pb.Tpo .deps/libmesos_no_3rdparty_la-mesos.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.pb.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Tpo -c -o scheduler/libmesos_no_3rdparty_la-scheduler.pb.lo `test -f 'scheduler/scheduler.pb.cc' || echo '../../src/'`s
 cheduler/scheduler.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.pb.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Tpo -c scheduler/scheduler.pb.cc  -fPIC -DPIC -o scheduler/.libs/libmesos_no_3rdparty_la-scheduler.pb.o
mv -f scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Tpo scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registry.pb.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registry.pb.Tpo -c -o master/libmesos_no_3rdparty_la-registry.pb.lo `test -f 'master/registry.pb.cc' || echo '../../src/'`master/registry.p
 b.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registry.pb.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registry.pb.Tpo -c master/registry.pb.cc  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-registry.pb.o
mv -f master/.deps/libmesos_no_3rdparty_la-registry.pb.Tpo master/.deps/libmesos_no_3rdparty_la-registry.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src    -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins
 /jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -DBUILD_DATE="\"$(date '+%Y-%m-%d %H:%M:%S')\"" -DBUILD_TIME="\"$(date '+%s')\"" -DBUILD_USER="\"$USER\"" -DBUILD_JAVA_JVM_LIBRARY=\"/home/jenkins/tools/java/jdk1.6.0_20-64/jre/lib/amd64/server/libjvm.so\"  -DBUILD_FLAGS="\"$BUILD_FLAGS\"" -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++
 11 -MT common/libbuild_la-build.lo -MD -MP -MF common/.deps/libbuild_la-build.Tpo -c -o common/libbuild_la-build.lo `test -f 'common/build.cpp' || echo '../../src/'`common/build.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src "-DBUILD_DATE=\"2015-02-10 15:04:06\"" -DBUILD_TIME=\"1423580646\" -DBUILD_USER=\"jenkins\" -DBUILD_JAVA_JVM_LIBRARY=\"/home/jenkins/tools/java/jdk1.6.0_20-64/jre/lib/amd64/server/libjvm.so\" -DBUILD_FLAGS=\"\" -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libbuild_la-build.lo -MD -MP -MF common/.deps/lib
 build_la-build.Tpo -c ../../src/common/build.cpp  -fPIC -DPIC -o common/.libs/libbuild_la-build.o
mv -f common/.deps/libbuild_la-build.Tpo common/.deps/libbuild_la-build.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/liblog_la-log.pb.lo -MD -MP -MF messages/.deps/liblog_la-log.pb.Tpo -c -o messages/liblog_la-log.pb.lo `test -f 'messages/log.pb.cc' || echo '../../src/'`messages/log.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/liblog_la-log.pb.lo -MD -MP -MF messages/.deps/liblog_la-log.pb.Tpo -c messages/log.pb.cc  -fPIC -DPIC -o messages/.libs/liblog_la-log.pb.o
mv -f slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Tpo slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/libstate_la-state.pb.lo -MD -MP -MF messages/.deps/libstate_la-state.pb.Tpo -c -o messages/libstate_la-state.pb.lo `test -f 'messages/state.pb.cc' || echo '../../src/'`messages/state.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/libstate_la-state.pb.lo -MD -MP -MF messages/.deps/libstate_la-state.pb.Tpo -c messages/state.pb.cc  -fPIC -DPIC -o messages/.libs/libstate_la-state.pb.o
mv -f messages/.deps/liblog_la-log.pb.Tpo messages/.deps/liblog_la-log.pb.Plo
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT local/mesos_local-main.o -MD -MP -MF local/.deps/mesos_local-main.Tpo -c -o local/mesos_local-main.o `test -f 'local/main.cpp' || echo '../../src/'`local/main.cpp
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Plo
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT log/mesos_log-main.o -MD -MP -MF log/.deps/mesos_log-main.Tpo -c -o log/mesos_log-main.o `test -f 'log/main.cpp' || echo '../../src/'`log/main.cpp
mv -f messages/.deps/libstate_la-state.pb.Tpo messages/.deps/libstate_la-state.pb.Plo
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/mesos/mesos_containerizer-main.o -MD -MP -MF slave/containerizer/mesos/.deps/mesos_containerizer-main.Tpo -c -o slave/containerizer/mesos/mesos_containerizer-main.o `test -f 'slave/containerizer/mesos/main.cpp' || echo '../../src/'`slave/containerizer/mesos/main.
 cpp
mv -f log/.deps/mesos_log-main.Tpo log/.deps/mesos_log-main.Po
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT usage/mesos_usage-main.o -MD -MP -MF usage/.deps/mesos_usage-main.Tpo -c -o usage/mesos_usage-main.o `test -f 'usage/main.cpp' || echo '../../src/'`usage/main.cpp
mv -f slave/containerizer/mesos/.deps/mesos_containerizer-main.Tpo slave/containerizer/mesos/.deps/mesos_containerizer-main.Po
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/mesos_master-main.o -MD -MP -MF master/.deps/mesos_master-main.Tpo -c -o master/mesos_master-main.o `test -f 'master/main.cpp' || echo '../../src/'`master/main.cpp
mv -f usage/.deps/mesos_usage-main.Tpo usage/.deps/mesos_usage-main.Po
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/mesos_slave-main.o -MD -MP -MF slave/.deps/mesos_slave-main.Tpo -c -o slave/mesos_slave-main.o `test -f 'slave/main.cpp' || echo '../../src/'`slave/main.cpp
mv -f local/.deps/mesos_local-main.Tpo local/.deps/mesos_local-main.Po
test "../.." = ".." ||			\
		(/bin/mkdir -p python/src/mesos && cp -pf ../../src/python/src/mesos/__init__.py python/src/mesos/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos && cp -pf ../../src/python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos/interface && cp -pf ../../src/python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos && cp -pf ../../src/python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/module.cpp python/native/src/mesos/native/module.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/module.hpp python/native/src/mesos/native/module.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_executor.cpp python/native/src/mesos/native/proxy_executor.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_executor.hpp python/native/src/mesos/native/proxy_executor.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.cpp python/native/src/mesos/native/proxy_scheduler.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.hpp python/native/src/mesos/native/proxy_scheduler.hpp)
Building protobuf Python egg ...
cd ../3rdparty/libprocess/3rdparty/protobuf-2.5.0/python && \
          CC="gcc" CXX="g++" CFLAGS="-g1 -O0 -Wno-unused-local-typedefs" CXXFLAGS="-g1 -O0 -Wno-unused-local-typedefs -std=c++11" \
          PYTHONPATH=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/distribute-0.6.26 /usr/bin/python setup.py build bdist_egg
running build
running build_py
Generating google/protobuf/descriptor_pb2.py...
Generating google/protobuf/compiler/plugin_pb2.py...
Generating google/protobuf/unittest_pb2.py...
Generating google/protobuf/unittest_custom_options_pb2.py...
Generating google/protobuf/unittest_import_pb2.py...
Generating google/protobuf/unittest_import_public_pb2.py...
Generating google/protobuf/unittest_mset_pb2.py...
Generating google/protobuf/unittest_no_generic_services_pb2.py...
Generating google/protobuf/internal/test_bad_identifiers_pb2.py...
Generating google/protobuf/internal/more_extensions_pb2.py...
Generating google/protobuf/internal/more_extensions_dynamic_pb2.py...
Generating google/protobuf/internal/more_messages_pb2.py...
Generating google/protobuf/internal/factory_test1_pb2.py...
Generating google/protobuf/internal/factory_test2_pb2.py...
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/google
creating build/lib.linux-x86_64-2.7/google/protobuf
creating build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/__init__.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/api_implementation.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/containers.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/cpp_message.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/decoder.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/encoder.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/enum_type_wrapper.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/message_listener.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/python_message.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/type_checkers.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/wire_format.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/__init__.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor_pb2.py -> build/lib.linux-x86_64-2.7/google/protobuf
creating build/lib.linux-x86_64-2.7/google/protobuf/compiler
copying google/protobuf/compiler/__init__.py -> build/lib.linux-x86_64-2.7/google/protobuf/compiler
copying google/protobuf/compiler/plugin_pb2.py -> build/lib.linux-x86_64-2.7/google/protobuf/compiler
copying google/protobuf/message.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor_database.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor_pool.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/message_factory.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/reflection.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/service.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/service_reflection.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/text_format.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/__init__.py -> build/lib.linux-x86_64-2.7/google
running bdist_egg
running egg_info
creating protobuf.egg-info
writing requirements to protobuf.egg-info/requires.txt
writing protobuf.egg-info/PKG-INFO
writing namespace_packages to protobuf.egg-info/namespace_packages.txt
writing top-level names to protobuf.egg-info/top_level.txt
writing dependency_links to protobuf.egg-info/dependency_links.txt
writing requirements to protobuf.egg-info/requires.txt
writing protobuf.egg-info/PKG-INFO
writing namespace_packages to protobuf.egg-info/namespace_packages.txt
writing top-level names to protobuf.egg-info/top_level.txt
writing dependency_links to protobuf.egg-info/dependency_links.txt
writing manifest file 'protobuf.egg-info/SOURCES.txt'
reading manifest file 'protobuf.egg-info/SOURCES.txt'
writing manifest file 'protobuf.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/google
copying build/lib.linux-x86_64-2.7/google/__init__.py -> build/bdist.linux-x86_64/egg/google
creating build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/service.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/service_reflection.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/message_factory.py -> build/bdist.linux-x86_64/egg/google/protobuf
creating build/bdist.linux-x86_64/egg/google/protobuf/compiler
copying build/lib.linux-x86_64-2.7/google/protobuf/compiler/plugin_pb2.py -> build/bdist.linux-x86_64/egg/google/protobuf/compiler
copying build/lib.linux-x86_64-2.7/google/protobuf/compiler/__init__.py -> build/bdist.linux-x86_64/egg/google/protobuf/compiler
copying build/lib.linux-x86_64-2.7/google/protobuf/reflection.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/__init__.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor_pb2.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/text_format.py -> build/bdist.linux-x86_64/egg/google/protobuf
creating build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/api_implementation.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/enum_type_wrapper.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/type_checkers.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/decoder.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/message_listener.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/cpp_message.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/__init__.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/containers.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/encoder.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/wire_format.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/python_message.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor_database.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/message.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor_pool.py -> build/bdist.linux-x86_64/egg/google/protobuf
byte-compiling build/bdist.linux-x86_64/egg/google/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/service.py to service.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/service_reflection.py to service_reflection.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/message_factory.py to message_factory.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/compiler/plugin_pb2.py to plugin_pb2.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/compiler/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/reflection.py to reflection.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_pb2.py to descriptor_pb2.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/text_format.py to text_format.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/api_implementation.py to api_implementation.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/enum_type_wrapper.py to enum_type_wrapper.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/type_checkers.py to type_checkers.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/decoder.py to decoder.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/message_listener.py to message_listener.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/cpp_message.py to cpp_message.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/containers.py to containers.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/encoder.py to encoder.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/wire_format.py to wire_format.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/python_message.py to python_message.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor.py to descriptor.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_database.py to descriptor_database.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/message.py to message.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_pool.py to descriptor_pool.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/namespace_packages.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/protobuf-2.5.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authentication/cram_md5/libmesos_no_3rdparty_la-auxprop.lo -MD -MP -MF authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Tpo -c -o authentication/cram_md5/libmesos_no_3rdparty_la-auxprop.lo `test -f 'authentication/cram_md
 5/auxprop.cpp' || echo '../../src/'`authentication/cram_md5/auxprop.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authentication/cram_md5/libmesos_no_3rdparty_la-auxprop.lo -MD -MP -MF authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Tpo -c ../../src/authentication/cram_md5/auxprop.cpp  -fPIC -DPIC -o authentication/cram_md5/.libs/libmesos_no_3rdparty_la-auxprop.o
mv -f authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Tpo authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authorizer/libmesos_no_3rdparty_la-authorizer.lo -MD -MP -MF authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Tpo -c -o authorizer/libmesos_no_3rdparty_la-authorizer.lo `test -f 'authorizer/authorizer.cpp' || echo '../../src/'`auth
 orizer/authorizer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authorizer/libmesos_no_3rdparty_la-authorizer.lo -MD -MP -MF authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Tpo -c ../../src/authorizer/authorizer.cpp  -fPIC -DPIC -o authorizer/.libs/libmesos_no_3rdparty_la-authorizer.o
mv -f master/.deps/mesos_master-main.Tpo master/.deps/mesos_master-main.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-attributes.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-attributes.Tpo -c -o common/libmesos_no_3rdparty_la-attributes.lo `test -f 'common/attributes.cpp' || echo '../../src/'`common/attributes.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-attributes.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-attributes.Tpo -c ../../src/common/attributes.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-attributes.o
mv -f authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Tpo authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-date_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-date_utils.Tpo -c -o common/libmesos_no_3rdparty_la-date_utils.lo `test -f 'common/date_utils.cpp' || echo '../../src/'`common/date_utils.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-date_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-date_utils.Tpo -c ../../src/common/date_utils.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-date_utils.o
mv -f common/.deps/libmesos_no_3rdparty_la-date_utils.Tpo common/.deps/libmesos_no_3rdparty_la-date_utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-http.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-http.Tpo -c -o common/libmesos_no_3rdparty_la-http.lo `test -f 'common/http.cpp' || echo '../../src/'`common/http.cpp
mv -f common/.deps/libmesos_no_3rdparty_la-attributes.Tpo common/.deps/libmesos_no_3rdparty_la-attributes.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-lock.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-lock.Tpo -c -o common/libmesos_no_3rdparty_la-lock.lo `test -f 'common/lock.cpp' || echo '../../src/'`common/lock.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-http.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-http.Tpo -c ../../src/common/http.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-http.o
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-lock.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-lock.Tpo -c ../../src/common/lock.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-lock.o
mv -f common/.deps/libmesos_no_3rdparty_la-lock.Tpo common/.deps/libmesos_no_3rdparty_la-lock.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-protobuf_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Tpo -c -o common/libmesos_no_3rdparty_la-protobuf_utils.lo `test -f 'common/protobuf_utils.cpp' || echo '../../src/'`comm
 on/protobuf_utils.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-protobuf_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Tpo -c ../../src/common/protobuf_utils.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-protobuf_utils.o
mv -f slave/.deps/mesos_slave-main.Tpo slave/.deps/mesos_slave-main.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-resources.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-resources.Tpo -c -o common/libmesos_no_3rdparty_la-resources.lo `test -f 'common/resources.cpp' || echo '../../src/'`common/resources.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-resources.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-resources.Tpo -c ../../src/common/resources.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-resources.o
mv -f common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Tpo common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-thread.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-thread.Tpo -c -o common/libmesos_no_3rdparty_la-thread.lo `test -f 'common/thread.cpp' || echo '../../src/'`common/thread.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-thread.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-thread.Tpo -c ../../src/common/thread.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-thread.o
mv -f common/.deps/libmesos_no_3rdparty_la-thread.Tpo common/.deps/libmesos_no_3rdparty_la-thread.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-type_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-type_utils.Tpo -c -o common/libmesos_no_3rdparty_la-type_utils.lo `test -f 'common/type_utils.cpp' || echo '../../src/'`common/type_utils.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-type_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-type_utils.Tpo -c ../../src/common/type_utils.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-type_utils.o
mv -f common/.deps/libmesos_no_3rdparty_la-http.Tpo common/.deps/libmesos_no_3rdparty_la-http.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-values.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-values.Tpo -c -o common/libmesos_no_3rdparty_la-values.lo `test -f 'common/values.cpp' || echo '../../src/'`common/values.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-values.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-values.Tpo -c ../../src/common/values.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-values.o
mv -f common/.deps/libmesos_no_3rdparty_la-resources.Tpo common/.deps/libmesos_no_3rdparty_la-resources.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT docker/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF docker/.deps/libmesos_no_3rdparty_la-docker.Tpo -c -o docker/libmesos_no_3rdparty_la-docker.lo `test -f 'docker/docker.cpp' || echo '../../src/'`docker/docker.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT docker/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF docker/.deps/libmesos_no_3rdparty_la-docker.Tpo -c ../../src/docker/docker.cpp  -fPIC -DPIC -o docker/.libs/libmesos_no_3rdparty_la-docker.o
mv -f common/.deps/libmesos_no_3rdparty_la-type_utils.Tpo common/.deps/libmesos_no_3rdparty_la-type_utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT exec/libmesos_no_3rdparty_la-exec.lo -MD -MP -MF exec/.deps/libmesos_no_3rdparty_la-exec.Tpo -c -o exec/libmesos_no_3rdparty_la-exec.lo `test -f 'exec/exec.cpp' || echo '../../src/'`exec/exec.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT exec/libmesos_no_3rdparty_la-exec.lo -MD -MP -MF exec/.deps/libmesos_no_3rdparty_la-exec.Tpo -c ../../src/exec/exec.cpp  -fPIC -DPIC -o exec/.libs/libmesos_no_3rdparty_la-exec.o
mv -f common/.deps/libmesos_no_3rdparty_la-values.Tpo common/.deps/libmesos_no_3rdparty_la-values.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT files/libmesos_no_3rdparty_la-files.lo -MD -MP -MF files/.deps/libmesos_no_3rdparty_la-files.Tpo -c -o files/libmesos_no_3rdparty_la-files.lo `test -f 'files/files.cpp' || echo '../../src/'`files/files.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT files/libmesos_no_3rdparty_la-files.lo -MD -MP -MF files/.deps/libmesos_no_3rdparty_la-files.Tpo -c ../../src/files/files.cpp  -fPIC -DPIC -o files/.libs/libmesos_no_3rdparty_la-files.o
mv -f files/.deps/libmesos_no_3rdparty_la-files.Tpo files/.deps/libmesos_no_3rdparty_la-files.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT hook/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF hook/.deps/libmesos_no_3rdparty_la-manager.Tpo -c -o hook/libmesos_no_3rdparty_la-manager.lo `test -f 'hook/manager.cpp' || echo '../../src/'`hook/manager.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT hook/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF hook/.deps/libmesos_no_3rdparty_la-manager.Tpo -c ../../src/hook/manager.cpp  -fPIC -DPIC -o hook/.libs/libmesos_no_3rdparty_la-manager.o
mv -f exec/.deps/libmesos_no_3rdparty_la-exec.Tpo exec/.deps/libmesos_no_3rdparty_la-exec.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT local/libmesos_no_3rdparty_la-local.lo -MD -MP -MF local/.deps/libmesos_no_3rdparty_la-local.Tpo -c -o local/libmesos_no_3rdparty_la-local.lo `test -f 'local/local.cpp' || echo '../../src/'`local/local.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT local/libmesos_no_3rdparty_la-local.lo -MD -MP -MF local/.deps/libmesos_no_3rdparty_la-local.Tpo -c ../../src/local/local.cpp  -fPIC -DPIC -o local/.libs/libmesos_no_3rdparty_la-local.o
mv -f docker/.deps/libmesos_no_3rdparty_la-docker.Tpo docker/.deps/libmesos_no_3rdparty_la-docker.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging/libmesos_no_3rdparty_la-logging.lo -MD -MP -MF logging/.deps/libmesos_no_3rdparty_la-logging.Tpo -c -o logging/libmesos_no_3rdparty_la-logging.lo `test -f 'logging/logging.cpp' || echo '../../src/'`logging/logging.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging/libmesos_no_3rdparty_la-logging.lo -MD -MP -MF logging/.deps/libmesos_no_3rdparty_la-logging.Tpo -c ../../src/logging/logging.cpp  -fPIC -DPIC -o logging/.libs/libmesos_no_3rdparty_la-logging.o
mv -f logging/.deps/libmesos_no_3rdparty_la-logging.Tpo logging/.deps/libmesos_no_3rdparty_la-logging.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-contender.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-contender.Tpo -c -o master/libmesos_no_3rdparty_la-contender.lo `test -f 'master/contender.cpp' || echo '../../src/'`master/contender.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-contender.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-contender.Tpo -c ../../src/master/contender.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-contender.o
mv -f hook/.deps/libmesos_no_3rdparty_la-manager.Tpo hook/.deps/libmesos_no_3rdparty_la-manager.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-constants.Tpo -c -o master/libmesos_no_3rdparty_la-constants.lo `test -f 'master/constants.cpp' || echo '../../src/'`master/constants.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-constants.Tpo -c ../../src/master/constants.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-constants.o
mv -f master/.deps/libmesos_no_3rdparty_la-constants.Tpo master/.deps/libmesos_no_3rdparty_la-constants.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-detector.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-detector.Tpo -c -o master/libmesos_no_3rdparty_la-detector.lo `test -f 'master/detector.cpp' || echo '../../src/'`master/detector.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-detector.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-detector.Tpo -c ../../src/master/detector.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-detector.o
mv -f local/.deps/libmesos_no_3rdparty_la-local.Tpo local/.deps/libmesos_no_3rdparty_la-local.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-drf_sorter.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-drf_sorter.Tpo -c -o master/libmesos_no_3rdparty_la-drf_sorter.lo `test -f 'master/drf_sorter.cpp' || echo '../../src/'`master/drf_sorter.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-drf_sorter.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-drf_sorter.Tpo -c ../../src/master/drf_sorter.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-drf_sorter.o
mv -f master/.deps/libmesos_no_3rdparty_la-contender.Tpo master/.deps/libmesos_no_3rdparty_la-contender.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-http.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-http.Tpo -c -o master/libmesos_no_3rdparty_la-http.lo `test -f 'master/http.cpp' || echo '../../src/'`master/http.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-http.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-http.Tpo -c ../../src/master/http.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-http.o
mv -f master/.deps/libmesos_no_3rdparty_la-drf_sorter.Tpo master/.deps/libmesos_no_3rdparty_la-drf_sorter.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-master.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-master.Tpo -c -o master/libmesos_no_3rdparty_la-master.lo `test -f 'master/master.cpp' || echo '../../src/'`master/master.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-master.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-master.Tpo -c ../../src/master/master.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-master.o
mv -f master/.deps/libmesos_no_3rdparty_la-detector.Tpo master/.deps/libmesos_no_3rdparty_la-detector.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c -o master/libmesos_no_3rdparty_la-metrics.lo `test -f 'master/metrics.cpp' || echo '../../src/'`master/metrics.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c ../../src/master/metrics.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-metrics.o
mv -f master/.deps/libmesos_no_3rdparty_la-http.Tpo master/.deps/libmesos_no_3rdparty_la-http.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registrar.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registrar.Tpo -c -o master/libmesos_no_3rdparty_la-registrar.lo `test -f 'master/registrar.cpp' || echo '../../src/'`master/registrar.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registrar.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registrar.Tpo -c ../../src/master/registrar.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-registrar.o
mv -f master/.deps/libmesos_no_3rdparty_la-metrics.Tpo master/.deps/libmesos_no_3rdparty_la-metrics.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-repairer.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-repairer.Tpo -c -o master/libmesos_no_3rdparty_la-repairer.lo `test -f 'master/repairer.cpp' || echo '../../src/'`master/repairer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-repairer.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-repairer.Tpo -c ../../src/master/repairer.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-repairer.o
mv -f master/.deps/libmesos_no_3rdparty_la-repairer.Tpo master/.deps/libmesos_no_3rdparty_la-repairer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-validation.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-validation.Tpo -c -o master/libmesos_no_3rdparty_la-validation.lo `test -f 'master/validation.cpp' || echo '../../src/'`master/validation.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-validation.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-validation.Tpo -c ../../src/master/validation.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-validation.o
mv -f master/.deps/libmesos_no_3rdparty_la-registrar.Tpo master/.deps/libmesos_no_3rdparty_la-registrar.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT module/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF module/.deps/libmesos_no_3rdparty_la-manager.Tpo -c -o module/libmesos_no_3rdparty_la-manager.lo `test -f 'module/manager.cpp' || echo '../../src/'`module/manager.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT module/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF module/.deps/libmesos_no_3rdparty_la-manager.Tpo -c ../../src/module/manager.cpp  -fPIC -DPIC -o module/.libs/libmesos_no_3rdparty_la-manager.o
mv -f master/.deps/libmesos_no_3rdparty_la-validation.Tpo master/.deps/libmesos_no_3rdparty_la-validation.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-constants.Tpo -c -o sched/libmesos_no_3rdparty_la-constants.lo `test -f 'sched/constants.cpp' || echo '../../src/'`sched/constants.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-constants.Tpo -c ../../src/sched/constants.cpp  -fPIC -DPIC -o sched/.libs/libmesos_no_3rdparty_la-constants.o
mv -f module/.deps/libmesos_no_3rdparty_la-manager.Tpo module/.deps/libmesos_no_3rdparty_la-manager.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-sched.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-sched.Tpo -c -o sched/libmesos_no_3rdparty_la-sched.lo `test -f 'sched/sched.cpp' || echo '../../src/'`sched/sched.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-sched.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-sched.Tpo -c ../../src/sched/sched.cpp  -fPIC -DPIC -o sched/.libs/libmesos_no_3rdparty_la-sched.o
mv -f sched/.deps/libmesos_no_3rdparty_la-constants.Tpo sched/.deps/libmesos_no_3rdparty_la-constants.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Tpo -c -o scheduler/libmesos_no_3rdparty_la-scheduler.lo `test -f 'scheduler/scheduler.cpp' || echo '../../src/'`scheduler/sc
 heduler.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Tpo -c ../../src/scheduler/scheduler.cpp  -fPIC -DPIC -o scheduler/.libs/libmesos_no_3rdparty_la-scheduler.o
mv -f master/.deps/libmesos_no_3rdparty_la-master.Tpo master/.deps/libmesos_no_3rdparty_la-master.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-constants.Tpo -c -o slave/libmesos_no_3rdparty_la-constants.lo `test -f 'slave/constants.cpp' || echo '../../src/'`slave/constants.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-constants.Tpo -c ../../src/slave/constants.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-constants.o
mv -f slave/.deps/libmesos_no_3rdparty_la-constants.Tpo slave/.deps/libmesos_no_3rdparty_la-constants.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-gc.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-gc.Tpo -c -o slave/libmesos_no_3rdparty_la-gc.lo `test -f 'slave/gc.cpp' || echo '../../src/'`slave/gc.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-gc.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-gc.Tpo -c ../../src/slave/gc.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-gc.o
mv -f slave/.deps/libmesos_no_3rdparty_la-gc.Tpo slave/.deps/libmesos_no_3rdparty_la-gc.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-graceful_shutdown.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Tpo -c -o slave/libmesos_no_3rdparty_la-graceful_shutdown.lo `test -f 'slave/graceful_shutdown.cpp' || echo '../../sr
 c/'`slave/graceful_shutdown.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-graceful_shutdown.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Tpo -c ../../src/slave/graceful_shutdown.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-graceful_shutdown.o
mv -f scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Tpo scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-http.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-http.Tpo -c -o slave/libmesos_no_3rdparty_la-http.lo `test -f 'slave/http.cpp' || echo '../../src/'`slave/http.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-http.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-http.Tpo -c ../../src/slave/http.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-http.o
mv -f sched/.deps/libmesos_no_3rdparty_la-sched.Tpo sched/.deps/libmesos_no_3rdparty_la-sched.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c -o slave/libmesos_no_3rdparty_la-metrics.lo `test -f 'slave/metrics.cpp' || echo '../../src/'`slave/metrics.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c ../../src/slave/metrics.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-metrics.o
mv -f slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Tpo slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-monitor.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-monitor.Tpo -c -o slave/libmesos_no_3rdparty_la-monitor.lo `test -f 'slave/monitor.cpp' || echo '../../src/'`slave/monitor.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-monitor.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-monitor.Tpo -c ../../src/slave/monitor.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-monitor.o
mv -f slave/.deps/libmesos_no_3rdparty_la-metrics.Tpo slave/.deps/libmesos_no_3rdparty_la-metrics.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-paths.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-paths.Tpo -c -o slave/libmesos_no_3rdparty_la-paths.lo `test -f 'slave/paths.cpp' || echo '../../src/'`slave/paths.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-paths.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-paths.Tpo -c ../../src/slave/paths.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-paths.o
mv -f slave/.deps/libmesos_no_3rdparty_la-http.Tpo slave/.deps/libmesos_no_3rdparty_la-http.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-state.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-state.Tpo -c -o slave/libmesos_no_3rdparty_la-state.lo `test -f 'slave/state.cpp' || echo '../../src/'`slave/state.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-state.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-state.Tpo -c ../../src/slave/state.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-state.o
mv -f slave/.deps/libmesos_no_3rdparty_la-paths.Tpo slave/.deps/libmesos_no_3rdparty_la-paths.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-slave.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-slave.Tpo -c -o slave/libmesos_no_3rdparty_la-slave.lo `test -f 'slave/slave.cpp' || echo '../../src/'`slave/slave.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-slave.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-slave.Tpo -c ../../src/slave/slave.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-slave.o
mv -f slave/.deps/libmesos_no_3rdparty_la-monitor.Tpo slave/.deps/libmesos_no_3rdparty_la-monitor.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-containerizer.lo `test -f 'slave/containeri
 zer/containerizer.cpp' || echo '../../src/'`slave/containerizer/containerizer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Tpo -c ../../src/slave/containerizer/containerizer.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-container
 izer.o
mv -f slave/.deps/libmesos_no_3rdparty_la-state.Tpo slave/.deps/libmesos_no_3rdparty_la-state.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-composing.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-composing.lo `test -f 'slave/containerizer/composin
 g.cpp' || echo '../../src/'`slave/containerizer/composing.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-composing.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Tpo -c ../../src/slave/containerizer/composing.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-composing.o
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-docker.lo `test -f 'slave/containerizer/docker.cpp' || ec
 ho '../../src/'`slave/containerizer/docker.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Tpo -c ../../src/slave/containerizer/docker.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-docker.o
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-external_containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-external_containerizer.lo
  `test -f 'slave/containerizer/external_containerizer.cpp' || echo '../../src/'`slave/containerizer/external_containerizer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-external_containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Tpo -c ../../src/slave/containerizer/external_containerizer.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmes
 os_no_3rdparty_la-external_containerizer.o
mv -f slave/.deps/libmesos_no_3rdparty_la-slave.Tpo slave/.deps/libmesos_no_3rdparty_la-slave.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-fetcher.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo `test -f 'slave/containerizer/fetcher.cpp' |
 | echo '../../src/'`slave/containerizer/fetcher.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-fetcher.Tpo -c ../../src/slave/containerizer/fetcher.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-fetcher.o
../../src/slave/containerizer/fetcher.cpp: In function 'Try<Bytes> mesos::slave::fetchSize(const string&, const string&)':
../../src/slave/containerizer/fetcher.cpp:367:23: error: 'lstatsize' is not a member of 'os'
     Try<Bytes> size = os::lstatsize(path.get());
                       ^
../../src/slave/containerizer/fetcher.cpp: In member function 'process::Future<Nothing> mesos::slave::FetcherProcess::reserveCacheSpace(const Bytes&, const string&)':
../../src/slave/containerizer/fetcher.cpp:457:34: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for (int i = 0; i < cache.size(); i++) {
                                  ^
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-isolator.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-isolator.lo `test -f 'slave/containerizer/isolator.cp
 p' || echo '../../src/'`slave/containerizer/isolator.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-isolator.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Tpo -c ../../src/slave/containerizer/isolator.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-isolator.o
../../src/slave/containerizer/fetcher.cpp: In member function 'off_t mesos::slave::FetcherProcess::completeCacheFile(const std::shared_ptr<mesos::slave::FetcherProcess::CacheFile>&, const string&)':
../../src/slave/containerizer/fetcher.cpp:993:21: error: 'lstatsize' is not a member of 'os'
   Try<Bytes> size = os::lstatsize(path);
                     ^
cc1plus: all warnings being treated as errors
make[3]: *** [slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Plo
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Plo
make[3]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
make: *** [distcheck] Error 1

- Mesos ReviewBot


On Feb. 10, 2015, 9:14 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2015, 9:14 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review72822
-----------------------------------------------------------


Bad patch!

Reviews applied: [30609, 30606]

Failed command: ./support/apply-review.sh -n -r 30606

Error:
 2015-02-17 23:14:27 URL:https://reviews.apache.org/r/30606/diff/raw/ [1463/1463] -> "30606.patch" [1]
error: patch failed: 3rdparty/libprocess/3rdparty/stout/include/stout/net.hpp:83
error: 3rdparty/libprocess/3rdparty/stout/include/stout/net.hpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Feb. 17, 2015, 4:40 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 4:40 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 24, 2015, 11:26 p.m., Timothy Chen wrote:
> > docs/fetcher.md, line 16
> > <https://reviews.apache.org/r/30774/diff/17/?file=872869#file872869line16>
> >
> >     The framework provides a list of CommandInfo::URI values as part of the TaskInfo message that becomes the input to the Mesos fetcher.

Even though you do not mean this, your suggested text could be misread that the entire TaskInfo message becomes input to the fetcher. Now proposing: 

As part of a TaskInfo message, the framework ordering the task's execution provides a list of
CommandInfo::URI protobuf values, which becomes the input to the Mesos fetcher.


> On Feb. 24, 2015, 11:26 p.m., Timothy Chen wrote:
> > docs/fetcher.md, line 68
> > <https://reviews.apache.org/r/30774/diff/17/?file=872869#file872869line68>
> >
> >     Not sure if putting the struct here is a good idea, as it's most likely going to be changed in the future.

When the struct changes, we change the doc.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73991
-----------------------------------------------------------


On Feb. 23, 2015, 3:33 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 23, 2015, 3:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 24, 2015, 11:26 p.m., Timothy Chen wrote:
> > docs/fetcher.md, line 68
> > <https://reviews.apache.org/r/30774/diff/17/?file=872869#file872869line68>
> >
> >     Not sure if putting the struct here is a good idea, as it's most likely going to be changed in the future.
> 
> Bernd Mathiske wrote:
>     When the struct changes, we change the doc.
> 
> Timothy Chen wrote:
>     We've avoided doing so in all our docs, and we also suggested removing from another review for the serice discovery docs.
>     No committer is going to remember this and help enforce keeping two places as the source of truth. If you really like to provide it leave a reference where the actual proto lives and comment and refer folks to there for the true defintions.

Yes, just putting the proto there is not good if we tend to avoid that, which I was not aware of. Thanks! In this case, it is very hard to explain what is going on without proto, so I would very much like to opt for your way out, to put a reference in place that points from the proto to the doc and vice-versa.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73991
-----------------------------------------------------------


On March 18, 2015, 11:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.

> On Feb. 25, 2015, 7:26 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 16
> > <https://reviews.apache.org/r/30774/diff/17/?file=872869#file872869line16>
> >
> >     The framework provides a list of CommandInfo::URI values as part of the TaskInfo message that becomes the input to the Mesos fetcher.
> 
> Bernd Mathiske wrote:
>     Even though you do not mean this, your suggested text could be misread that the entire TaskInfo message becomes input to the fetcher. Now proposing: 
>     
>     As part of a TaskInfo message, the framework ordering the task's execution provides a list of
>     CommandInfo::URI protobuf values, which becomes the input to the Mesos fetcher.

Sure, just wanted to make the wording more understandable


> On Feb. 25, 2015, 7:26 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 68
> > <https://reviews.apache.org/r/30774/diff/17/?file=872869#file872869line68>
> >
> >     Not sure if putting the struct here is a good idea, as it's most likely going to be changed in the future.
> 
> Bernd Mathiske wrote:
>     When the struct changes, we change the doc.

We've avoided doing so in all our docs, and we also suggested removing from another review for the serice discovery docs.
No committer is going to remember this and help enforce keeping two places as the source of truth. If you really like to provide it leave a reference where the actual proto lives and comment and refer folks to there for the true defintions.


- Timothy


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73991
-----------------------------------------------------------


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73991
-----------------------------------------------------------



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120418>

    loosely



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120419>

    The framework provides a list of CommandInfo::URI values as part of the TaskInfo message that becomes the input to the Mesos fetcher.



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120420>

    same URI leads to downloading another copy of the same resource.



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120421>

    What's file size queries to the cache?



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120422>

    Not sure if putting the struct here is a good idea, as it's most likely going to be changed in the future.



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120423>

    the affected task



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120424>

    per-user basis


- Timothy Chen


On Feb. 23, 2015, 11:33 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 23, 2015, 11:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74010
-----------------------------------------------------------


Bad patch!

Reviews applied: [30609, 30606, 30774]

Failed command: ./support/apply-review.sh -n -r 30774

Error:
 2015-02-25 10:26:35 URL:https://reviews.apache.org/r/30774/diff/raw/ [159918/159918] -> "30774.patch" [1]
error: patch failed: src/Makefile.am:1301
error: src/Makefile.am: patch does not apply
error: patch failed: src/launcher/fetcher.cpp:34
error: src/launcher/fetcher.cpp: patch does not apply
error: patch failed: src/slave/containerizer/fetcher.hpp:20
error: src/slave/containerizer/fetcher.hpp: patch does not apply
error: patch failed: src/slave/containerizer/fetcher.cpp:16
error: src/slave/containerizer/fetcher.cpp: patch does not apply
error: patch failed: src/tests/fetcher_tests.cpp:56
error: src/tests/fetcher_tests.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Feb. 25, 2015, 10:19 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 10:19 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > This really is an impressive project Bernd, tipping my hat here!
> > 
> > I did not get all the way through, had to ignore the tests in this review (to be continued!).
> > 
> > For playing a bit with this and for testing the functionality, it would be great if this was rebased to the current master.
> > 
> > Some high level comments; 
> > I kind of like the idea of wrapping synchronous calls into async's, thus allowing error cases to fall through then-cases and hence enabling an error handling that isnt scattered all over. While you are now able to concentrate the failure handling, there is a cost involved; 1. it becomes harder to read, 2. it possibly becomes more prone to unforseen concurrency issues, 3. this launches one-off processes a lot lot lot :).

1. Compared to what alternative?
2. Compared to what alternative?
3. "Launching" one-off processes should not be a problem if we have our thread pool implemented well, which I am assuming. Are you referring to the many async() calls? This stuff is supposed to be cheap and available :-)


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.hpp, line 122
> > <https://reviews.apache.org/r/30774/diff/18/?file=875546#file875546line122>
> >
> >     Why are we exposing the FetchProcess definition within this header? In other words, does anyone outside the Fetcher implementation need to know about this process' symbols?

Yes: tests.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, line 1073
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line1073>
> >
> >     Why using `auto` here - is this sanctioned by our styleguide?

I have 2 ship its for the style guide change. Please commit it :-)

https://reviews.apache.org/r/30792/


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, line 1084
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line1084>
> >
> >     Clang-format seems to prefer not to indend a function name after its return value at all. I have never encountered such case before, hence I am entirely unsure about the mesos (aka __right__) way.
> >     
> >     This appears to be triggered mostly by the fact that you are using nested class definitions - I would try to avoid that unless your implementation and the definitions are in one place as the resulting symbols are really hard to identify on a single glance.

If I don't nest them they need longer names. That saves me "::" per class name and leaves more room for interpretation. We are not using Clang-format, so this is a non-issue. (I am convinced we should move to an automatic formatter yesterday, for the record).


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/launcher/fetcher.cpp, line 138
> > <https://reviews.apache.org/r/30774/diff/18/?file=875542#file875542line138>
> >
> >     Why are you mentioning `os::net` to the user?

Carried this over from old fetcher code. Removed now.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/launcher/fetcher.cpp, line 236
> > <https://reviews.apache.org/r/30774/diff/18/?file=875542#file875542line236>
> >
> >     Not yours but this sounds weird, what do you think?

It now says:

  // We regard as local: "file://" or the absense of any URI scheme.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/launcher/fetcher.cpp, lines 425-426
> > <https://reviews.apache.org/r/30774/diff/18/?file=875542#file875542line425>
> >
> >     Is this part of the comment an artifact of older revisions? The exit code does not seem to correlate with the number of fetched items.

Yep. I will bring this back later.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.hpp, lines 19-20
> > <https://reviews.apache.org/r/30774/diff/18/?file=875546#file875546line19>
> >
> >     I know this particular issue has not yet gotten a final vote, but I would suggest using 
> >     `__SLAVE_CONTAINERIZER_FETCHER_HPP__` instead.

Done.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, lines 148-151
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line148>
> >
> >     Can we get rid of this validation here or within `validateUri`?
> >     
> >     In any case, please make sure the resulting Error messages are consistent.

Carried over from old fetcher code. Now compressed into one place. :-)


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, lines 236-239
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line236>
> >
> >     Are these URIs canonicalized towards lower case already at some point?
> >     
> >     According to the standard, a mixed case URI / scheme is perfectly valid: `Http://example.org/example.tar.gz`.
> >     
> >     See http://tools.ietf.org/html/std66#section-3.1 again :).

I just carried over how it used to be. Wanna file an improvement ticket?


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, line 358
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line358>
> >
> >     Couldnt T get deduced from the async returned container (Try<T>) -- in other words, do you __need__ to call `tryToFuture<Nothing>(..)` instead of letting the compiler select the right one for you (`tryToFuture(..)`)?

My C++ compiler is not smart enough for this and rejects it.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, lines 496-502
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line496>
> >
> >     How about:
> >     
> >     ```
> >     .then(defer(
> >         self(),
> >         &FetcherProcess::__fetch,
> >         containerId,
> >         lambda::_1,
> >         ...
> >     ```
> >     
> >     or 
> >     
> >     ```
> >     .then(defer(self(),
> >                 &FetcherProcess::__fetch,
> >                 containerId,
> >                 lambda::_1,
> >                 ...
> >     ```
> >     ?

Fixed.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, line 527
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line527>
> >
> >     Would you mind adding a comment on how could we possibly end up here with non existing paths?

I cannot find out what line of code you are referring to. RB does not show me a line number or some source code that matches. If you mean how we can end up with a deletion failure, this could be due to hardware failure as reported by a disk driver, for example.


> On Feb. 25, 2015, 2:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, line 809
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line809>
> >
> >     This appears to be a use case for `discard` as you are no longer interested in the results and, if possible, wish to abort any actions, right? 
> >     
> >     Such discard however certainly needs to be taken care of within your entire chain (which currently only addresses `onFailure` but not `onDiscard`).

The action has already happened here. We are merely reporting now that it has failed.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73462
-----------------------------------------------------------


On Feb. 25, 2015, 9:54 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 9:54 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Till Toenshoff <to...@me.com>.

> On Feb. 25, 2015, 10:20 p.m., Till Toenshoff wrote:
> > src/slave/containerizer/fetcher.cpp, lines 459-460
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line459>
> >
> >     So a force removes from cache -- can you explain why this is needed?

Ow I got it :) ... a force is a "remove -> re/download -> cache -> extract / copy"


- Till


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73462
-----------------------------------------------------------


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Till Toenshoff <to...@me.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73462
-----------------------------------------------------------


This really is an impressive project Bernd, tipping my hat here!

I did not get all the way through, had to ignore the tests in this review (to be continued!).

For playing a bit with this and for testing the functionality, it would be great if this was rebased to the current master.

Some high level comments; 
I kind of like the idea of wrapping synchronous calls into async's, thus allowing error cases to fall through then-cases and hence enabling an error handling that isnt scattered all over. While you are now able to concentrate the failure handling, there is a cost involved; 1. it becomes harder to read, 2. it possibly becomes more prone to unforseen concurrency issues, 3. this launches one-off processes a lot lot lot :).


docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120473>

    s/reponsible/responsible/



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120474>

    s/specifiy/specify/



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120475>

    s/somehwere/somewhere/



include/mesos/mesos.proto
<https://reviews.apache.org/r/30774/#comment120476>

    This should be part of the URI message definition as it is not used by the CommandInfo message directly.



src/Makefile.am
<https://reviews.apache.org/r/30774/#comment120462>

    Alphabetize please.



src/hdfs/hdfs.hpp
<https://reviews.apache.org/r/30774/#comment120477>

    For debugging purposes, it might be nice to see the actual result from this HDFS callup. Maybe something like:
    ```
    return Error("HDFS du returned unexpected number of results: HDFS returned '" << output.str() << "'");
    ```



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120478>

    Why are you mentioning `os::net` to the user?



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120468>

    Not yours but this sounds weird, what do you think?



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120469>

    It should be enough to return the error and let the upper layers decide on the logging, no?



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120470>

    The extract method itself does not explicitely return the source path in the error message. It will however contain that information by the shell call of `extract` (lines 73 + 79):
    ```
    command += " '" + sourcePath + "'";
    ```
    
    ```
    return Error("Failed to extract: command " + command +
                 " exited with status: " + stringify(status));
    ```
    
    For consistency, I would suggest to either getting rid of his very extension of the error message or to adding the same in line 237.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120471>

    How about moving this up and thereby getting rid of the brackets around the "expected" case?



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120472>

    Is this part of the comment an artifact of older revisions? The exit code does not seem to correlate with the number of fetched items.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment120490>

    I know this particular issue has not yet gotten a final vote, but I would suggest using 
    `__SLAVE_CONTAINERIZER_FETCHER_HPP__` instead.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment120491>

    Why are we exposing the FetchProcess definition within this header? In other words, does anyone outside the Fetcher implementation need to know about this process' symbols?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120493>

    Why not making this a `std::string` as you never use it in its raw, C-style format anyway?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120494>

    See above.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120495>

    Schemes should not be limited to a length (<= 5), the standard does not mention such restriction (see http://tools.ietf.org/html/std66#section-3.1).



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120497>

    Can we get rid of this validation here or within `validateUri`?
    
    In any case, please make sure the resulting Error messages are consistent.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120621>

    Are these URIs canonicalized towards lower case already at some point?
    
    According to the standard, a mixed case URI / scheme is perfectly valid: `Http://example.org/example.tar.gz`.
    
    See http://tools.ietf.org/html/std66#section-3.1 again :).



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120487>

    Hah, this is neat :).



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120500>

    Couldnt T get deduced from the async returned container (Try<T>) -- in other words, do you __need__ to call `tryToFuture<Nothing>(..)` instead of letting the compiler select the right one for you (`tryToFuture(..)`)?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120488>

    So a force removes from cache -- can you explain why this is needed?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120548>

    How about:
    
    ```
    .then(defer(
        self(),
        &FetcherProcess::__fetch,
        containerId,
        lambda::_1,
        ...
    ```
    
    or 
    
    ```
    .then(defer(self(),
                &FetcherProcess::__fetch,
                containerId,
                lambda::_1,
                ...
    ```
    ?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120549>

    This continuation should IMHO be indended the same way the above `then` is. It otherwise becomes a bit messy, I feel.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120596>

    Would you mind adding a comment on how could we possibly end up here with non existing paths?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120553>

    `fetcherInfo.add_items()->CopyFrom(item)` ?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120501>

    This appears to be a use case for `discard` as you are no longer interested in the results and, if possible, wish to abort any actions, right? 
    
    Such discard however certainly needs to be taken care of within your entire chain (which currently only addresses `onFailure` but not `onDiscard`).



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120569>

    Why using `auto` here - is this sanctioned by our styleguide?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120570>

    Clang-format seems to prefer not to indend a function name after its return value at all. I have never encountered such case before, hence I am entirely unsure about the mesos (aka __right__) way.
    
    This appears to be triggered mostly by the fact that you are using nested class definitions - I would try to avoid that unless your implementation and the definitions are in one place as the resulting symbols are really hard to identify on a single glance.



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment120482>

    "TODO(bernd-mesos): Add a ..." ?



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment120483>

    "// Make sure we start ..." ?


- Till Toenshoff


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74069
-----------------------------------------------------------


Bad patch!

Reviews applied: [30609, 30606, 30774]

Failed command: ./support/apply-review.sh -n -r 30774

Error:
 2015-02-25 19:43:38 URL:https://reviews.apache.org/r/30774/diff/raw/ [160364/160364] -> "30774.patch" [1]
error: patch failed: src/Makefile.am:1301
error: src/Makefile.am: patch does not apply
error: patch failed: src/launcher/fetcher.cpp:34
error: src/launcher/fetcher.cpp: patch does not apply
error: patch failed: src/slave/containerizer/fetcher.hpp:20
error: src/slave/containerizer/fetcher.hpp: patch does not apply
error: patch failed: src/slave/containerizer/fetcher.cpp:16
error: src/slave/containerizer/fetcher.cpp: patch does not apply
error: patch failed: src/tests/fetcher_tests.cpp:56
error: src/tests/fetcher_tests.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 10:56 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 528
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line528>
> >
> >     return size.error();

That does not compile.


> On Feb. 25, 2015, 10:56 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 832
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line832>
> >
> >     Can you leave a high level comments on the top, talking about how reserve, set space, victims, eviction is designed to work together?

I added an overview of how eviction works at the top of reserveCacheSpace.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74053
-----------------------------------------------------------


On Feb. 25, 2015, 9:54 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 9:54 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 10:56 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 528
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line528>
> >
> >     return size.error();
> 
> Bernd Mathiske wrote:
>     That does not compile.
> 
> Timothy Chen wrote:
>     It doesn't? How about wrapping in Error again?
>     And why is returning size correct here?

Yes, we can wrap it in Error again. If we don't it does not compile because a string does not match a Try. I will wrap it so it does not stand out.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74053
-----------------------------------------------------------


On March 18, 2015, 11:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.

> On Feb. 25, 2015, 6:56 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 528
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line528>
> >
> >     return size.error();
> 
> Bernd Mathiske wrote:
>     That does not compile.

It doesn't? How about wrapping in Error again?
And why is returning size correct here?


- Timothy


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74053
-----------------------------------------------------------


On Feb. 26, 2015, 2:03 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 26, 2015, 2:03 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp fed0105946da579a38357a30e7ae56e646e05b89 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp d290f95251def3952c5ee34f600e1d71467f6293 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 2438620e2db94c3c56336fa5d8e69a18fe8f3bac 
>   src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
>   src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74053
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120556>

    return size.error();



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120555>

    Can you leave a high level comments on the top, talking about how reserve, set space, victims, eviction is designed to work together?


- Timothy Chen


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 10:25 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 103
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line103>
> >
> >     How about introducing a Cache namespace, so then we can just do Fetcher::Cache::recover, Fetcher::Cache::clean, etc?

There is already a cache class. The shape of this code is an artefact of not injecting slaveId and flags. We will clean this up when we refactor for the latter.


> On Feb. 25, 2015, 10:25 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 516
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line516>
> >
> >     Shouldn't you also log the error too?

This probably does not interrupt or change any user-driven action. It might, but it also might not.


> On Feb. 25, 2015, 10:25 a.m., Timothy Chen wrote:
> > src/slave/flags.hpp, line 108
> > <https://reviews.apache.org/r/30774/diff/19/?file=875884#file875884line108>
> >
> >     So the default value is actually not a recommended practice? Why set the default to that then?

So we can run tests. I have adjusted the doc as indicated above.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74043
-----------------------------------------------------------


On Feb. 25, 2015, 9:54 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 9:54 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74043
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120534>

    How about introducing a Cache namespace, so then we can just do Fetcher::Cache::recover, Fetcher::Cache::clean, etc?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120537>

    Shouldn't you also log the error too?



src/slave/flags.hpp
<https://reviews.apache.org/r/30774/#comment120530>

    So the default value is actually not a recommended practice? Why set the default to that then?


- Timothy Chen


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 10:30 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 686
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line686>
> >
> >     Seems like lookupEntry is only used here, and it's always coupled with a reference call.
> >     
> >     How about making the API less error prone, we can increment the reference directly in lookupEntry (also we should call it getEntry, as don't usually use lookup in our code base).
> >     
> >     so getEntry becomes ->
> >     - get cache key
> >     - get cache item
> >     - if cache item is present, increment ref
> >     - return item
> >     
> >     This way no one needs to call reference().

Renamed to getEntry(). 

I disagree with the plan to combine looking up entries and referencing them. I think this would make the code more error-prone, not less. It is too easy then to use getEntry() and inadvertently also reference() when someone later changes the code base. 

We would then better rename it to getAndReferenceEntry(). However, this does not increase code readability at all.

BTW, getEntry is also used in one other location, but I could inline it there, so we are back to one place.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74048
-----------------------------------------------------------


On Feb. 25, 2015, 9:54 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 9:54 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 10:30 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 686
> > <https://reviews.apache.org/r/30774/diff/19/?file=875881#file875881line686>
> >
> >     Seems like lookupEntry is only used here, and it's always coupled with a reference call.
> >     
> >     How about making the API less error prone, we can increment the reference directly in lookupEntry (also we should call it getEntry, as don't usually use lookup in our code base).
> >     
> >     so getEntry becomes ->
> >     - get cache key
> >     - get cache item
> >     - if cache item is present, increment ref
> >     - return item
> >     
> >     This way no one needs to call reference().
> 
> Bernd Mathiske wrote:
>     Renamed to getEntry(). 
>     
>     I disagree with the plan to combine looking up entries and referencing them. I think this would make the code more error-prone, not less. It is too easy then to use getEntry() and inadvertently also reference() when someone later changes the code base. 
>     
>     We would then better rename it to getAndReferenceEntry(). However, this does not increase code readability at all.
>     
>     BTW, getEntry is also used in one other location, but I could inline it there, so we are back to one place.

In the latest revision I changed it to two different methods. One is called getEntry() and does not reference and the other is called referenceEntry() and does. So we can have it both ways.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74048
-----------------------------------------------------------


On March 18, 2015, 11:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74048
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120547>

    Seems like lookupEntry is only used here, and it's always coupled with a reference call.
    
    How about making the API less error prone, we can increment the reference directly in lookupEntry (also we should call it getEntry, as don't usually use lookup in our code base).
    
    so getEntry becomes ->
    - get cache key
    - get cache item
    - if cache item is present, increment ref
    - return item
    
    This way no one needs to call reference().


- Timothy Chen


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Till Toenshoff <to...@me.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74155
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120678>

    Why not including the error message received by `chown`?
    
    ```
    return Error("Failed to chown directory " + directory + ": " + chown.error());
    ```


- Till Toenshoff


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74278
-----------------------------------------------------------


Patch looks great!

Reviews applied: [30606, 30609, 30774]

All tests passed.

- Mesos ReviewBot


On Feb. 26, 2015, 2:03 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 26, 2015, 2:03 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp fed0105946da579a38357a30e7ae56e646e05b89 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp d290f95251def3952c5ee34f600e1d71467f6293 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 2438620e2db94c3c56336fa5d8e69a18fe8f3bac 
>   src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
>   src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74446
-----------------------------------------------------------


Patch looks great!

Reviews applied: [30606, 30609, 30774]

All tests passed.

- Mesos ReviewBot


On Feb. 27, 2015, 3:58 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 27, 2015, 3:58 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 0ae0a10b41a2c0f7459c771b31c76bbc0c02df4f 
>   src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
>   src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74703
-----------------------------------------------------------


Patch looks great!

Reviews applied: [30606, 30609, 30774]

All tests passed.

- Mesos ReviewBot


On March 1, 2015, 3:42 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 1, 2015, 3:42 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 2, 2015, 8:13 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 759
> > <https://reviews.apache.org/r/30774/diff/25/?file=882228#file882228line759>
> >
> >     If you're incrementing all the time just to count, why not just get the size from list?
> 
> Bernd Mathiske wrote:
>     I am not incrementing to count anything. I am incrementing to hit the right index in a vector that parallels the list I am iterating over. Is there a C++ or Boost construct that can do this without indices?

Switched to using a const_iterator for this. This should be more obviously paralleling the foreach.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74885
-----------------------------------------------------------


On March 7, 2015, 7:21 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 7:21 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 2, 2015, 8:13 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 759
> > <https://reviews.apache.org/r/30774/diff/25/?file=882228#file882228line759>
> >
> >     If you're incrementing all the time just to count, why not just get the size from list?

I am not incrementing to count anything. I am incrementing to hit the right index in a vector that parallels the list I am iterating over. Is there a C++ or Boost construct that can do this without indices?


> On March 2, 2015, 8:13 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 831
> > <https://reviews.apache.org/r/30774/diff/25/?file=882228#file882228line831>
> >
> >     Why is the check entries necessary? Seems like if this for test only we should do the validations in test?

This is "in tests". This method is for testing. It says so in its header file comment.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74885
-----------------------------------------------------------


On March 3, 2015, 5:01 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 5:01 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   docs/images/fetch_cache.jpg PRE-CREATION 
>   docs/images/fetch_components.jpg PRE-CREATION 
>   docs/images/fetch_flow.jpg PRE-CREATION 
>   docs/images/fetch_force1.jpg PRE-CREATION 
>   docs/images/fetch_force2.jpg PRE-CREATION 
>   docs/images/fetch_state.jpg PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74885
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment121710>

    If you're incrementing all the time just to count, why not just get the size from list?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment121711>

    Why is the check entries necessary? Seems like if this for test only we should do the validations in test?


- Timothy Chen


On March 2, 2015, 6:27 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 2, 2015, 6:27 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   docs/images/fetch_components.jpg PRE-CREATION 
>   docs/images/fetch_flow.jpg PRE-CREATION 
>   docs/images/fetch_state.jpg PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 4, 2015, 4:39 p.m., Jay Buffington wrote:
> > Hey Bernd,
> > 
> > I'm really looking forward to this feature.  There's a lot here, so I was hoping you could help me understand by responding to some of these questions:
> > 
> > Why do you need the cache table data structure?  Just use the filesystem?
> > Why are the expanded files cached as well?  
> > There shouldn’t be different behavior if we’re using the cache.  My understaing is that with this patch, if we use the cache the tar doesn’t exist in the sandbox.  Isn't this a regression?
> > What’s the point of segregating the cache by user?
> > Why not respect http caching headers?
> > Why does the framework need to even know if the cache is in use or not?
> > The images referenced in the fetcher docs aren’t part of the review.  Where can I find them?
> > 
> > Thanks!
> > Jay

Hi Jay,

thanks for these great questions! In summary, everything you are asking for feature-wise can be offered later (soonish) by relatively simple to implement feature additions. 

Answers to your questions in order as follows.

- If I just used the file system to implement the cache without a libprocess actor as complement, I would need to persist state about cache contents, use file locks, coordinate multiple instances of running mesos-fetcher programs, etc. There is a possible alternative architecture for this that would also work. See the JIRA commoents on MESOS-336 for an earlier discussion on this. My personal preference would be to perhaps further develop what is now FetcherProcess into an external program (with fail-over) rather than trying to beef up mesos-fetcher, which would lead to a lot of IPC for coordination.
- I am not aware of caching expanded files. We only cache the archive file itself.
- Not having a tar file in the sandbox is not a regression if you see using the cache at all as a new feature. But I can copy it over optionally if so desired in an add-on patch. This is just MVP and it seems more likely that people would rather not have the tar file copy.
- I would not want to have a framework for one user plant a cache file that a framework of another user then picks up. This file could be lying around for a long time, from way before the second framework starts. We can later make this optional as an extra feature. I am erring on the side of caution in this MVP.
- Excellent suggestion. But this is for later. Extra feature that I also find important.
- We can have another URI.cache value that makes it so.
- Sorry for having removed the images for now. I had trouble applying the patch with pictures in it. Advice on what git/RB supports here is welcome! For now, you can git clone https://github.com/bernd-mesos/MesosFetcherDocs and then open the md files locally or you can look at the PDFs which I also uploaded.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75266
-----------------------------------------------------------


On March 5, 2015, 3:15 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 3:15 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Jay Buffington <me...@jaybuff.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75266
-----------------------------------------------------------


Hey Bernd,

I'm really looking forward to this feature.  There's a lot here, so I was hoping you could help me understand by responding to some of these questions:

Why do you need the cache table data structure?  Just use the filesystem?
Why are the expanded files cached as well?  
There shouldn’t be different behavior if we’re using the cache.  My understaing is that with this patch, if we use the cache the tar doesn’t exist in the sandbox.  Isn't this a regression?
What’s the point of segregating the cache by user?
Why not respect http caching headers?
Why does the framework need to even know if the cache is in use or not?
The images referenced in the fetcher docs aren’t part of the review.  Where can I find them?

Thanks!
Jay


docs/fetcher-cache-internals.md
<https://reviews.apache.org/r/30774/#comment122236>

    s/eactly/exactly/


- Jay Buffington


On March 4, 2015, 12:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 4, 2015, 12:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/launcher/fetcher.cpp, line 321
> > <https://reviews.apache.org/r/30774/diff/33/?file=888350#file888350line321>
> >
> >     Can you comment the relationship between the FetcherInfo::Item and the FetcherInfo here? Is the FetcherInfo::Item within the FetcherInfo but FetcherInfo is included because you just want to get the 'sandbox_directory' and 'cache_directory' and rather than pulling those out explicitly you just passed the entire FetcherInfo?

There are more items in the FetcherInfo than just the one we are working in here. That's why this one is called out explicitly. I changed this to passing both directories in.


> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/launcher/fetcher.cpp, lines 364-366
> > <https://reviews.apache.org/r/30774/diff/33/?file=888350#file888350line364>
> >
> >     Why are these not CHECKs? Since you're the one setting up the FetcherInfo it seems like you should know explicitly whether or not the cache_filename was set!
> >     
> >     Same for the cache_directory below as well.

What if somebody else uses mesos-fetcher?


> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/launcher/fetcher.cpp, lines 403-404
> > <https://reviews.apache.org/r/30774/diff/33/?file=888350#file888350line403>
> >
> >     As mentioned above, it would be great to really capture the relationship between the FetcherInfo and the FetcherInfo::Item. If The FetcherInfo encapsulates the FetcherInfo::Item I would also suggest switching the order of the parameters to signify that.

The main purpose here is to fetch this one particular item, not everything FetcherInfo carries. FetcherInfo is a secondary parameter that provides extra parameters like cache_directory, sandbox_directory, and framework_home. Putting it second makes this relationship clear IMHO. Do you suggest adding all these as individual parameters?

Yes, the item is included in the list of items in FetcherInfo. Shall we break up FetcherInfo into several shells, the inner one without items?


> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/slave/flags.hpp, line 487
> > <https://reviews.apache.org/r/30774/diff/33/?file=888358#file888358line487>
> >
> >     Can we make this a Path to start?

Then it would be the only one. Confusing. I'd rather have a wholesale sweep over the whole code base to introduce Path - as a separate ticket.


> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/slave/slave.cpp, line 796
> > <https://reviews.apache.org/r/30774/diff/33/?file=888359#file888359line796>
> >
> >     We should do recovery on the fetcher itself:
> >     
> >     Try<Nothing> recover = fetcher->recover(flags, slaveId);
> >     
> >     It seems very weird to have a static generic Fetcher recover functionality that implies that we can't have multiple Fetchers running at the same time. How do we start multiple slaves at the same time?

This is an artefact of the lack of injection of slaveId and flags. It should be cleaned up when we refactor those. The slave does not have access to the fetcher instance as it is right now. It would cause a lot of collateral changes if it did. I advise to refrain for now. I have put a comment at the static method to explain this. That's the best fix for now IMHO.

There is no problem starting multiple slaves, because they all have a different slaveID that gets passed into this call.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75754
-----------------------------------------------------------


On March 7, 2015, 7:21 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 7:21 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/launcher/fetcher.cpp, lines 364-366
> > <https://reviews.apache.org/r/30774/diff/33/?file=888350#file888350line364>
> >
> >     Why are these not CHECKs? Since you're the one setting up the FetcherInfo it seems like you should know explicitly whether or not the cache_filename was set!
> >     
> >     Same for the cache_directory below as well.
> 
> Bernd Mathiske wrote:
>     What if somebody else uses mesos-fetcher?

Adding a comment why this is not a check.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75754
-----------------------------------------------------------


On April 30, 2015, 7:40 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 30, 2015, 7:40 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 10, 2015, 8:35 a.m., Benjamin Hindman wrote:
> > src/launcher/fetcher.cpp, lines 403-404
> > <https://reviews.apache.org/r/30774/diff/33/?file=888350#file888350line403>
> >
> >     As mentioned above, it would be great to really capture the relationship between the FetcherInfo and the FetcherInfo::Item. If The FetcherInfo encapsulates the FetcherInfo::Item I would also suggest switching the order of the parameters to signify that.
> 
> Bernd Mathiske wrote:
>     The main purpose here is to fetch this one particular item, not everything FetcherInfo carries. FetcherInfo is a secondary parameter that provides extra parameters like cache_directory, sandbox_directory, and framework_home. Putting it second makes this relationship clear IMHO. Do you suggest adding all these as individual parameters?
>     
>     Yes, the item is included in the list of items in FetcherInfo. Shall we break up FetcherInfo into several shells, the inner one without items?

Now using split up parameters instead of FetcherInfo to pass around sandbox dir, cache dir, frameworks home.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75754
-----------------------------------------------------------


On April 30, 2015, 7:40 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 30, 2015, 7:40 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75754
-----------------------------------------------------------



include/mesos/fetcher/fetcher.proto
<https://reviews.apache.org/r/30774/#comment122998>

    First, the "preloading" feature does not exist so let's not comment as much. Second, if you perceive this being optional in the future we should make it be optional now.



include/mesos/mesos.proto
<https://reviews.apache.org/r/30774/#comment122995>

    Let's remove the 'force' functionality for now. Why? First, as a user this is a tedious primitive to use: it requires the scheduler to set the field when it wants a new version of the URI but there is no way for the scheduler to know after the new version has been properly cached on a particular machine. If a user always wants the latest version they should just set 'cache' to false. A better approach here would be to introduce a 'sha' for the URI that the fetcher can compare against and if the SHAs are different force the download then (but let's not do the SHA right now, we can do that as a follow up, and keep the MVP simple).
    
    Given that then we'll have a basic enum for 'Cache' and folks have already been confused by the values 'FETCH' and 'FORCE' I think we should revert to the simpler: 'optional bool cache'. What would be other potential 'enum Cache' values going forward? If we can think of good examples then the right thing to do here would probably be to introduce a 'message CacheInfo' that we embed in URI.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122999>

    Can we put a newline above this to make it easier to read? The code is very compressed here with the line continuation from 'sourcePath ='.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123000>

    Why not 'else if'? The value of doing 'else if' here is that this block is comparing the status of the same variable so continuing with an 'else if' helps to capture that semantics.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123001>

    Can we please add a TODO here to refactor this into stout so that people can more easily chmod an exectuable? For example, we could define some static flags so that someone can do:
    
    os::chmod(path, EXECUTABLE_CHMOD_FLAGS);



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123004>

    Can we improve the error message here (and below), e.g.:
    
    return Error("Failed to determine the basename of the URI: " + basename.error());



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123006>

    Can you comment the relationship between the FetcherInfo::Item and the FetcherInfo here? Is the FetcherInfo::Item within the FetcherInfo but FetcherInfo is included because you just want to get the 'sandbox_directory' and 'cache_directory' and rather than pulling those out explicitly you just passed the entire FetcherInfo?



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123012>

    Understanding the logic here is confusing:
    
    (1) Why do we copy the file if extracting failed? It's clear to me that this is because this is the way we've always done this, but with the new code pulled into this function we should capture this with comments for a future reader.
    
    (2) Why do we return the sandbox directory when extraction was sucessful? Why do we return the value from 'chmodExecutable'? I would have expected to test whether 'chmodExecuable' was successful and then returned whatever the semantics of this function should return. Let's comment these functions return semantics.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123025>

    Why are these not CHECKs? Since you're the one setting up the FetcherInfo it seems like you should know explicitly whether or not the cache_filename was set!
    
    Same for the cache_directory below as well.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123015>

    Newline above here to break this up from the complicated line above.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment123028>

    As mentioned above, it would be great to really capture the relationship between the FetcherInfo and the FetcherInfo::Item. If The FetcherInfo encapsulates the FetcherInfo::Item I would also suggest switching the order of the parameters to signify that.



src/slave/flags.hpp
<https://reviews.apache.org/r/30774/#comment123031>

    Can we make this a Path to start?



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment123202>

    We should do recovery on the fetcher itself:
    
    Try<Nothing> recover = fetcher->recover(flags, slaveId);
    
    It seems very weird to have a static generic Fetcher recover functionality that implies that we can't have multiple Fetchers running at the same time. How do we start multiple slaves at the same time?


- Benjamin Hindman


On March 7, 2015, 3:21 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 3:21 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 9, 2015, 8:37 a.m., Joerg Schad wrote:
> > src/tests/fetcher_cache_tests.cpp, line 134
> > <https://reviews.apache.org/r/30774/diff/33/?file=888361#file888361line134>
> >
> >     Can't we simulate SERIALIZED_TASK externally (as discussed)? In this we would not have several modes...

The whole ExecutionMode enum should go. We should use executeTask inside the loop that creates TaskInfos in each test and then wait explicitly inide or outside the loop as needed. I'll refactor accordingly in the next iteration. Also, we don't need enum value FAIL_TO_FETCH. It's not used anywhere any more.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75700
-----------------------------------------------------------


On March 7, 2015, 7:21 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 7:21 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 9, 2015, 8:37 a.m., Joerg Schad wrote:
> > include/mesos/mesos.proto, line 208
> > <https://reviews.apache.org/r/30774/diff/33/?file=888347#file888347line208>
> >
> >     Could you add a comment (i.e. backlink to the documention) reminding developers to update docs/fetcher.md when the protobuf is changed?

Since we are dropping the enum, there will be no such comment. There is one next to the remaining "cache" filed, though.


> On March 9, 2015, 8:37 a.m., Joerg Schad wrote:
> > src/slave/containerizer/fetcher.cpp, line 450
> > <https://reviews.apache.org/r/30774/diff/33/?file=888355#file888355line450>
> >
> >     size_t position?

Using an iterator now.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75700
-----------------------------------------------------------


On March 7, 2015, 7:21 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 7:21 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Joerg Schad <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75700
-----------------------------------------------------------



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment122920>

    Could we add an explicit Note about the current behavior that the cache does not consider changed/updated URIs (It is mentioned in the above paragraph but having an explicit note about this would be helpful)?



include/mesos/mesos.proto
<https://reviews.apache.org/r/30774/#comment122919>

    Could you add a comment (i.e. backlink to the documention) reminding developers to update docs/fetcher.md when the protobuf is changed?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122922>

    size_t position?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122921>

    Should we add a Check that items and uris have the same length in order to document this assumption explicitly?



src/slave/flags.hpp
<https://reviews.apache.org/r/30774/#comment122918>

    Could you add a description of these flags to configuration.md (i.e. http://mesos.apache.org/documentation/latest/configuration/)?



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment122924>

    Can't we simulate SERIALIZED_TASK externally (as discussed)? In this we would not have several modes...


- Joerg Schad


On March 7, 2015, 3:21 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 3:21 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77019
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124786>

    The error case is handled right after line 711, which closes the branch for non-error.


- Bernd Mathiske


On March 17, 2015, 6:59 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 17, 2015, 6:59 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 491
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line491>
> >
> >     Why not just mock _fetch and do a barrier on it by giving it a promise in test?
> 
> Bernd Mathiske wrote:
>     "just mock _fetch" is more work and harder to understand.
>     
>     It would also function, but then you would need to touch test code every time you change _fetch(). Furthermore, it would not be as clear why we wait for this particular call.

Meanwhile I tried mocking _fetch, but it does not work. See the related/duplicate issue below. Let's drop this one here now so we can keep the comments on the same topic and code region in one place going forward, OK?


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review76902
-----------------------------------------------------------


On April 10, 2015, 4:33 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 10, 2015, 4:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 3a8e8bf303e0576c212951f6028af77e54d93537 
>   include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
>   src/Makefile.am fa609da08e23d6595a3f6d2efddd3e333b6c78f1 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp 6893684e6d199a5d69fc8bba8e60c4acaae9c3c9 
>   src/slave/containerizer/docker.cpp f9fb07806e3b7d7d2afc1be3b8756eac23b32dcd 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp e4136095fca55637864f495098189ab3ad8d8fe7 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp 35f56252cfda5011d21aa188f33cc3e68a694968 
>   src/slave/slave.cpp 9fec023b643d410f4d511fa6f80e9835bab95b7e 
>   src/tests/docker_containerizer_tests.cpp c772d4c836de18b0e87636cb42200356d24ec73d 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 0e98572a62ae05437bd2bc800c370ad1a0c43751 
>   src/tests/mesos.cpp 02cbb4b8cf1206d0f32d160addc91d7e0f1ab28b 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 503
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line503>
> >
> >     Since this is only called in one place, how about put this in ___fetch, pass it the future and check if it failed log it there?
> 
> Bernd Mathiske wrote:
>     How would this be simpler and more readable?
>     
>     What is wrong with abstracting functions that are called only once? Doing so saves a comment / pulls what would have been a comment into code!

Since we probably don't need a comment here, I'll fix it.


> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 726
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line726>
> >
> >     Why ignore error?
> 
> Bernd Mathiske wrote:
>     The code that follows this line as of line 712 handles the error case.

See issue below for resolution.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review76902
-----------------------------------------------------------


On March 18, 2015, 11:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 406
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line406>
> >
> >     Do we call fetch even if we don't have anything to fetch? I think it will be a good idea to have a fast return if there is nothing to be fetched.

There is a check for this in Fetcher::fetch(). No need to even dispatch the call to the process either if there is nothing to fetch.


> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 491
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line491>
> >
> >     Why not just mock _fetch and do a barrier on it by giving it a promise in test?

"just mock _fetch" is more work and harder to understand.

It would also function, but then you would need to touch test code every time you change _fetch(). Furthermore, it would not be as clear why we wait for this particular call.


> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 503
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line503>
> >
> >     Since this is only called in one place, how about put this in ___fetch, pass it the future and check if it failed log it there?

How would this be simpler and more readable?

What is wrong with abstracting functions that are called only once? Doing so saves a comment / pulls what would have been a comment into code!


> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 518
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line518>
> >
> >     In what scenario should a cache entry not exist?
> >     If it doesn't somehow we won't be able to use it too?

As you can see at the call sites, this method is used in scenarios where fetching succeeded, where it failed, and incidentally where it left a partial download lying around. I added this comment:

  // We may or may not have started downloading. The download may or may
  // not have been partial. In any case, clean up whatever is there.
  
If there is no file, that's fine. Then we tried fetching and failed before starting to write the file. 

In any case, we remove the cache entry and the space amount it had reserved/claimed is released for later use.


> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 521
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line521>
> >
> >     Feel like this can be in a infintie loop, where if we can expire one item then forever other fetch items will get stuck?
> >     I wonder if we should have some remedy action, or simply crash too?

This is not a loop, because the cache entry gets removed BEFORE we attempt to delete the file. See line 500 just above.

However, just in case future changed code were ever to call this method several times on the same entry, I added a line that sets the entry's size field to zero. This way, accounted cache space is only released once.


> On March 18, 2015, 11:05 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 726
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line726>
> >
> >     Why ignore error?

The code that follows this line as of line 712 handles the error case.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review76902
-----------------------------------------------------------


On March 17, 2015, 6:59 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 17, 2015, 6:59 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review76902
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124598>

    What's the intention of this helper method? 
    Seems odd to only have one for HDFS and not local or Net. If none of this is shared, I say we can lump this all in fetchSize.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124605>

    Do we call fetch even if we don't have anything to fetch? I think it will be a good idea to have a fast return if there is nothing to be fetched.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124599>

    Why not just mock _fetch and do a barrier on it by giving it a promise in test?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124602>

    Since this is only called in one place, how about put this in ___fetch, pass it the future and check if it failed log it there?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124600>

    In what scenario should a cache entry not exist?
    If it doesn't somehow we won't be able to use it too?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124601>

    Feel like this can be in a infintie loop, where if we can expire one item then forever other fetch items will get stuck?
    I wonder if we should have some remedy action, or simply crash too?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124607>

    Why ignore error?


- Timothy Chen


On March 17, 2015, 1:59 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 17, 2015, 1:59 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 19, 2015, 9:40 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 379
> > <https://reviews.apache.org/r/30774/diff/38/?file=899705#file899705line379>
> >
> >     benh, what do you think of Bernd's contentionBarrier injection? commonly we always just mock the callback (_fetch in this case) in tests to block, but Bernd wanted to introduce a specific empty method for tests. I told him this is not a pattern we use in Mesos, but like to see what you think.
> 
> Bernd Mathiske wrote:
>     Of course I will stick to the prevalent patterns unless you start liking this one :-)

Turns out there is no method that offers itself opportunistically for the prevalent pattern. The next call up is AFTER waiting for the futures that the barrier needs to be BEFORE. Suggestions?

Alternatives (without judging them):
- Factor out the loop that gathers the futures and make it a method that gets called once. Then mock this method and have it dual-purposed as contention barrier. 
- Make the futures globally visible, await them in the test, too.
- Use an explicit lock instead of mocking.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77057
-----------------------------------------------------------


On March 24, 2015, 6:57 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 24, 2015, 6:57 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 19, 2015, 9:40 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 379
> > <https://reviews.apache.org/r/30774/diff/38/?file=899705#file899705line379>
> >
> >     benh, what do you think of Bernd's contentionBarrier injection? commonly we always just mock the callback (_fetch in this case) in tests to block, but Bernd wanted to introduce a specific empty method for tests. I told him this is not a pattern we use in Mesos, but like to see what you think.
> 
> Bernd Mathiske wrote:
>     Of course I will stick to the prevalent patterns unless you start liking this one :-)
> 
> Bernd Mathiske wrote:
>     Turns out there is no method that offers itself opportunistically for the prevalent pattern. The next call up is AFTER waiting for the futures that the barrier needs to be BEFORE. Suggestions?
>     
>     Alternatives (without judging them):
>     - Factor out the loop that gathers the futures and make it a method that gets called once. Then mock this method and have it dual-purposed as contention barrier. 
>     - Make the futures globally visible, await them in the test, too.
>     - Use an explicit lock instead of mocking.

Another option has been implemented in the latest patch: cut off the lower half of fetch() and call it _fetch(), then mock the latter. This variant ressembles existing other code the most.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77057
-----------------------------------------------------------


On April 30, 2015, 7:40 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 30, 2015, 7:40 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 19, 2015, 9:40 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 379
> > <https://reviews.apache.org/r/30774/diff/38/?file=899705#file899705line379>
> >
> >     benh, what do you think of Bernd's contentionBarrier injection? commonly we always just mock the callback (_fetch in this case) in tests to block, but Bernd wanted to introduce a specific empty method for tests. I told him this is not a pattern we use in Mesos, but like to see what you think.

Of course I will stick to the prevalent patterns unless you start liking this one :-)


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77057
-----------------------------------------------------------


On March 18, 2015, 11:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77057
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124854>

    benh, what do you think of Bernd's contentionBarrier injection? commonly we always just mock the callback (_fetch in this case) in tests to block, but Bernd wanted to introduce a specific empty method for tests. I told him this is not a pattern we use in Mesos, but like to see what you think.


- Timothy Chen


On March 19, 2015, 6:43 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 19, 2015, 6:43 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On April 12, 2015, 11:11 p.m., Timothy Chen wrote:
> > src/tests/fetcher_cache_tests.cpp, line 308
> > <https://reviews.apache.org/r/30774/diff/42/?file=922829#file922829line308>
> >
> >     Not sure why you picked an arbitrary number 5 here, why not let it be passed in?

OK, I will add an explanation in a comment. Two requirements need to be met by this constant.
- It needs to be larger than the expected number of status updates. We might choose something much larger than 5, but all tests run just fine with 5.
- It needs to be finite. Otherwise we will keep waiting for updates when none arrive due to a bug.

However, if we passed this constant in, then we would need to explain it at all the call sites, i.e. multiple times instead of only once. But the situation is exactly the same every time. So I will refrain from that.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review79838
-----------------------------------------------------------


On April 10, 2015, 4:33 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 10, 2015, 4:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 3a8e8bf303e0576c212951f6028af77e54d93537 
>   include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
>   src/Makefile.am fa609da08e23d6595a3f6d2efddd3e333b6c78f1 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp 6893684e6d199a5d69fc8bba8e60c4acaae9c3c9 
>   src/slave/containerizer/docker.cpp f9fb07806e3b7d7d2afc1be3b8756eac23b32dcd 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp e4136095fca55637864f495098189ab3ad8d8fe7 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp 35f56252cfda5011d21aa188f33cc3e68a694968 
>   src/slave/slave.cpp 9fec023b643d410f4d511fa6f80e9835bab95b7e 
>   src/tests/docker_containerizer_tests.cpp c772d4c836de18b0e87636cb42200356d24ec73d 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 0e98572a62ae05437bd2bc800c370ad1a0c43751 
>   src/tests/mesos.cpp 02cbb4b8cf1206d0f32d160addc91d7e0f1ab28b 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review79838
-----------------------------------------------------------



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment129437>

    Not sure why you picked an arbitrary number 5 here, why not let it be passed in?



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment129438>

    Always one file expected in the cache


- Timothy Chen


On April 10, 2015, 11:33 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 10, 2015, 11:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 3a8e8bf303e0576c212951f6028af77e54d93537 
>   include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
>   src/Makefile.am fa609da08e23d6595a3f6d2efddd3e333b6c78f1 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp 6893684e6d199a5d69fc8bba8e60c4acaae9c3c9 
>   src/slave/containerizer/docker.cpp f9fb07806e3b7d7d2afc1be3b8756eac23b32dcd 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp e4136095fca55637864f495098189ab3ad8d8fe7 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp 35f56252cfda5011d21aa188f33cc3e68a694968 
>   src/slave/slave.cpp 9fec023b643d410f4d511fa6f80e9835bab95b7e 
>   src/tests/docker_containerizer_tests.cpp c772d4c836de18b0e87636cb42200356d24ec73d 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 0e98572a62ae05437bd2bc800c370ad1a0c43751 
>   src/tests/mesos.cpp 02cbb4b8cf1206d0f32d160addc91d7e0f1ab28b 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On April 29, 2015, 1:46 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, lines 386-387
> > <https://reviews.apache.org/r/30774/diff/41/?file=904359#file904359line386>
> >
> >     How about still calling this s/reference/get/ and then checking the references? Also, we're deprecating and removing the return value of const &. Thus:
> >     
> >     Option<shared_ptr<Cache::Entry>>& entry =
> >       cache.get(commandUser, uri.value());
> >     
> >     if (entry.isSome()) {
> >       CHECK(entry.get()->references() > 0);
> >       ... 
> >     }

After talking about this we concluded to go with explicit referencing outside the accessor.


> On April 29, 2015, 1:46 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, lines 394-395
> > <https://reviews.apache.org/r/30774/diff/41/?file=904359#file904359line394>
> >
> >     Let's get a CHECK here too:
> >     
> >     CHECK(newEntry->references() > 0);

See above.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review78530
-----------------------------------------------------------


On April 30, 2015, 7:40 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 30, 2015, 7:40 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review78530
-----------------------------------------------------------



include/mesos/type_utils.hpp
<https://reviews.apache.org/r/30774/#comment127347>

    Let's also 'hash_combine' the 'extract' and 'executable' information here too.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment127351>

    Great TODO, let's point the comments in slave.cpp to this TODO please!



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment127352>

    This function no longer exists!



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment127353>

    +4 please!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127354>

    Not used!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127356>

    Two spaces between top-level definitions please!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127355>

    Please comment/TODO why this is being used and that you plan to remove this once we have C++11 lambdas!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127358>

    How about still calling this s/reference/get/ and then checking the references? Also, we're deprecating and removing the return value of const &. Thus:
    
    Option<shared_ptr<Cache::Entry>>& entry =
      cache.get(commandUser, uri.value());
    
    if (entry.isSome()) {
      CHECK(entry.get()->references() > 0);
      ... 
    }



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127357>

    +2 not +4



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127359>

    Let's get a CHECK here too:
    
    CHECK(newEntry->references() > 0);



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127360>

    Let's move this comment to the top of this function so that readers of the code have a better idea of what we're trying to accomplish sooner!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment127365>

    Let's add another comment to remind readers why if the entry's 'completion' is still pending this implies that we need to DOWNLOAD_AND_CACHE? For example:
    
    // Since the entry is not yet "complete", i.e., 'completion().isPending()', 
    // then it must be the case that we created the entry in
    // FetcherProcess::fetch otherwise the entry should have been
    // in the cache and we would have waited for the completion in
    // FetcherProcess::fetch.



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment127348>

    s/recoverCache/recover/ <-- Since in the future there might be other "recovery" things that need to get done that doesn't have anything to do with the cache.
    
    Also, let's please leave a comment here that explains why we're calling a static method rather than invoking a method on an instance of a 'Fetcher' directly. Our intuition is that this will likely have to change in the future.



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment127350>

    Comment here too, please, thanks!


- Benjamin Hindman


On April 29, 2015, 8:42 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 29, 2015, 8:42 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On April 29, 2015, 3:41 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, line 386
> > <https://reviews.apache.org/r/30774/diff/44/?file=945694#file945694line386>
> >
> >     Kill the const &.

Killed the &. Any reason this should not be const?


> On April 29, 2015, 3:41 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, line 597
> > <https://reviews.apache.org/r/30774/diff/44/?file=945694#file945694line597>
> >
> >     Let's add some helper functions on Fetcher::Cache so that we can just get this information directly in the tests rather than this "helper" function.
> >     
> >     // Return the cache.
> >     Try<list<Path>> FetcherProcess::cacheFiles();
> >     
> >     // Returns the number of entries in the cache.
> >     size_t FetcherProcess::cacheSize();

Can we please postpone this until after refactoring fetcher injection into slave/containerizer? It will be much easier to make these member functions then. I'll put a TODO for now.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review82017
-----------------------------------------------------------


On April 29, 2015, 1:42 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 29, 2015, 1:42 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On April 29, 2015, 3:41 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, lines 408-417
> > <https://reviews.apache.org/r/30774/diff/44/?file=945694#file945694line408>
> >
> >     For the future:
> >     
> >     auto futures = filter(entries, [](const auto& entry) { return entry.isSome() ? entry.get() : None(); });
> >     
> >     ;-)

I could not find a suitable filter function  in std, boost, or stout yet. Shall we create one?


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review82017
-----------------------------------------------------------


On May 12, 2015, 3:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 12, 2015, 3:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review82017
-----------------------------------------------------------



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment132684>

    const Bytes&
    
    ?



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment132638>

    No longer used function.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132642>

    Please replace tab with spaces.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132648>

    Kill the const &.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132657>

    Let's be explicit for now with references:
    
    Option<shared_ptr<Cache::Entry>> entry = cache.get();
    
    if (entry.isSome()) {
      entry.get()->reference();
      entries[uri] = entry.get()->completion()
        .then(defer(self(), [=]() { return entry.get(); });
    } else {
      shared_ptr<Cache::Entry> newEntry =
        cache.create(cacheDirectory, commandUser, uri);
    
      newEntry->reference();
    
      entries[uri] = async(&fetchSize, uri.value(), flags.frameworks_home)
        .then(defer(self(), 
    }



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132645>

    +2 not +4 here.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132646>

    // Wait for the URI to be downloaded into the cache (or fail).



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132643>

    .then(defer(self(), [=]() { return entry.get(); }))
    
    Then please kill the 'value' function above!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132688>

    // NOTE: We break this into two pieces because we want to be able to __block__ an instance of ...
    return _fetch(...);



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132658>

    For the future:
    
    auto futures = filter(entries, [](const auto& entry) { return entry.isSome() ? entry.get() : None(); });
    
    ;-)



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132666>

    Let's add some helper functions on Fetcher::Cache so that we can just get this information directly in the tests rather than this "helper" function.
    
    // Return the cache.
    Try<list<Path>> FetcherProcess::cacheFiles();
    
    // Returns the number of entries in the cache.
    size_t FetcherProcess::cacheSize();



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132667>

    CHECK_READY



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132669>

    Lambda-ify! ;-)



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132668>

    Please replaces tabs with spaces.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132670>

    Lambda-ify! ;-)



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132672>

    { on newline please.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment132671>

    CHECK_SOME



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment132632>

    Ideally we can inject the Fetcher instance into the Slave so that we don't have this global recover operation that is actually per slave:
    
    --------------------------------------------------
    
    Fetcher fetcher(flags);
    
    Slave slave(..., &fetcher);
    
    Slave::registered(...)
    {
      ...;
      Try<Nothing> recover = fetcher->recover(slaveid);
      if (recover.is...) {
        ...;
      }
      ...;
    }
    
    --------------------------------------------------
    
    But for now, let's just s/recoverCache/recover/ since the fact that the fetcher has a cache is an implementation detail.


- Benjamin Hindman


On April 29, 2015, 8:42 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 29, 2015, 8:42 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review83946
-----------------------------------------------------------


- Bernd Mathiske


On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:07 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On May 15, 2015, 10:34 a.m., Bernd Mathiske wrote:
> > src/tests/fetcher_cache_tests.cpp, line 426
> > <https://reviews.apache.org/r/30774/diff/48/?file=958588#file958588line426>
> >
> >     This needs to be a pointer. Or use a simple struct. Name: FetcherCacheTest::Task
> 
> Bernd Mathiske wrote:
>     Can't use the return struct approach, since Queue turns out not to be copyable. Going with passing in pointers.

Was looking at the wrong queue. Was able to use the return struct approach.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review83945
-----------------------------------------------------------


On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:07 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On May 15, 2015, 10:34 a.m., Bernd Mathiske wrote:
> > src/tests/fetcher_cache_tests.cpp, line 426
> > <https://reviews.apache.org/r/30774/diff/48/?file=958588#file958588line426>
> >
> >     This needs to be a pointer. Or use a simple struct. Name: FetcherCacheTest::Task

Can't use the return struct approach, since Queue turns out not to be copyable. Going with passing in pointers.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review83945
-----------------------------------------------------------


On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:07 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review83945
-----------------------------------------------------------



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment135004>

    This needs to be a pointer. Or use a simple struct. Name: FetcherCacheTest::Task


- Bernd Mathiske


On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:07 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > Just took a cursory glance since this is a huge diff, could it have been broken apart further? We've found large diffs like this one are next to impossible to review thoroughly :)

Agreed that this is normally the case. And this had been multiple smaller diffs (see "description" above). Benh eventually asked me to combine them for relatively long review sessions that covered a lot of ground in one swoop.


> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > src/slave/containerizer/fetcher.hpp, lines 288-295
> > <https://reviews.apache.org/r/30774/diff/52/?file=966644#file966644line288>
> >
> >     We might be able to get away with more descriptive names here to avoid the need for these comments:
> >     
> >     ```
> >     Bytes capacity;
> >     Bytes reserved;
> >     unsigned long fileCounter;
> >     ```
> >     
> >     'space' seems to suggest available space (to me), whereas 'capacity' seems pretty standard as a name for this. For 'tally', I can't tell from the name what is being tallied, but if we change the name to 'reserved' I have an understanding that this is the reserved space, not necessarily occupied but reserved for a purpose.

Much better indeed. Thanks!


> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > src/slave/containerizer/fetcher.hpp, lines 297-299
> > <https://reviews.apache.org/r/30774/diff/52/?file=966644#file966644line297>
> >
> >     Hard to tell why shared_ptr here is needed rather than Shared, or just Cache::Entry directly. Is there concurrent modification happening, or?

Shared is not mutable. Are you suggesting to exchange the whole entry every time we update a field?


> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > src/slave/containerizer/fetcher.cpp, line 617
> > <https://reviews.apache.org/r/30774/diff/52/?file=966645#file966645line617>
> >
> >     No need for the stringify here and below.

Thanks! will fix


> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > src/slave/containerizer/fetcher.cpp, lines 1131-1132
> > <https://reviews.apache.org/r/30774/diff/52/?file=966645#file966645line1131>
> >
> >     CHECK_LT will print the two numbers for you :)

Thanks! will fix


> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > src/slave/containerizer/fetcher.cpp, lines 1144-1145
> > <https://reviews.apache.org/r/30774/diff/52/?file=966645#file966645line1144>
> >
> >     Seems like an odd message format, since normally a meaning follows from a ':'
> >     
> >     ```
> >     Fetcher cache space overflow - space used: 2GB, exceeds total fetcher cache space: 1GB
> >     ```
> >     
> >     Here's another format where the meaning is described after the colon:
> >     
> >     ```
> >     Fetcher cache space overflow: 2GB used vs 1GB capacity
> >     ```

Yep, that's better.


> On June 5, 2015, 7:10 p.m., Ben Mahler wrote:
> > src/slave/containerizer/mesos/containerizer.hpp, lines 183-188
> > <https://reviews.apache.org/r/30774/diff/52/?file=966646#file966646line183>
> >
> >     I can't tell why slave id is being passed here, is there something subtle going on?

Yes. The slaveId is needed to create per-slave cache directories. There are multiple comments about this in other places that explain this and how this will go away when we will inject the slaveId after some refactoring. I will add a comment here as well.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review86871
-----------------------------------------------------------


On May 21, 2015, 9:05 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 21, 2015, 9:05 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
>   include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
>   src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
>   src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
>   src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
>   src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
>   src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
>   src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
>   src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
>   src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review86871
-----------------------------------------------------------


Just took a cursory glance since this is a huge diff, could it have been broken apart further? We've found large diffs like this one are next to impossible to review thoroughly :)


src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment139000>

    We might be able to get away with more descriptive names here to avoid the need for these comments:
    
    ```
    Bytes capacity;
    Bytes reserved;
    unsigned long fileCounter;
    ```
    
    'space' seems to suggest available space (to me), whereas 'capacity' seems pretty standard as a name for this. For 'tally', I can't tell from the name what is being tallied, but if we change the name to 'reserved' I have an understanding that this is the reserved space, not necessarily occupied but reserved for a purpose.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment139052>

    Hard to tell why shared_ptr here is needed rather than Shared, or just Cache::Entry directly. Is there concurrent modification happening, or?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment139051>

    No need for the stringify here and below.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment139053>

    CHECK_LT will print the two numbers for you :)



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment139055>

    Seems like an odd message format, since normally a meaning follows from a ':'
    
    ```
    Fetcher cache space overflow - space used: 2GB, exceeds total fetcher cache space: 1GB
    ```
    
    Here's another format where the meaning is described after the colon:
    
    ```
    Fetcher cache space overflow: 2GB used vs 1GB capacity
    ```



src/slave/containerizer/mesos/containerizer.hpp
<https://reviews.apache.org/r/30774/#comment139050>

    I can't tell why slave id is being passed here, is there something subtle going on?


- Ben Mahler


On May 21, 2015, 4:05 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 21, 2015, 4:05 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
>   include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
>   src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
>   src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
>   src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
>   src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
>   src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
>   src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
>   src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
>   src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review87615
-----------------------------------------------------------



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment140008>

    Any reason not using AWAIT_READY? CHECK_READY will abort the process while AWAIT_READY will just abort the test.
    
    ```
    19:53:22 DEBUG: F0611 19:53:23.743101 60646 fetcher_cache_tests.cpp:354] CHECK_READY(offers): is PENDING Failed to wait for resource offers
    19:53:22 DEBUG: *** Check failure stack trace: ***
    19:53:22 DEBUG:     @     0x7f431c3a644d  google::LogMessage::Fail()
    19:53:22 DEBUG:     @     0x7f431c3a828d  google::LogMessage::SendToLog()
    19:53:22 DEBUG:     @     0x7f431c3a603c  google::LogMessage::Flush()
    19:53:22 DEBUG:     @     0x7f431c3a8b89  google::LogMessageFatal::~LogMessageFatal()
    19:53:22 DEBUG:     @           0x53d9b8  _CheckFatal::~_CheckFatal()
    19:53:22 DEBUG:     @           0x66c26f  mesos::internal::tests::FetcherCacheTest::launchTask()
    19:53:22 DEBUG:     @           0x66fb09  mesos::internal::tests::FetcherCacheTest_CachedFallback_Test::TestBody()
    19:53:22 DEBUG:     @           0xbb1db3  testing::internal::HandleExceptionsInMethodIfSupported<>()
    19:53:22 DEBUG:     @           0xba9057  testing::Test::Run()
    19:53:22 DEBUG:     @           0xba90fe  testing::TestInfo::Run()
    19:53:22 DEBUG:     @           0xba9205  testing::TestCase::Run()
    19:53:22 DEBUG:     @           0xba94a8  testing::internal::UnitTestImpl::RunAllTests()
    19:53:22 DEBUG:     @           0xba9747  testing::UnitTest::Run()
    19:53:22 DEBUG:     @           0x4a1dc3  main
    19:53:22 DEBUG:     @     0x7f431a1d7d5d  __libc_start_main
    19:53:22 DEBUG:     @           0x4ad109  (unknown)
    19:53:23 DEBUG: make[3]: *** [check-local] Aborted (core dumped)
    19:53:23 DEBUG: make[3]: Leaving directory `/builddir/build/BUILD/mesos-0.23.0/src'
    19:53:23 DEBUG: make[2]: *** [check-am] Error 2
    19:53:23 DEBUG: make[2]: Leaving directory `/builddir/build/BUILD/mesos-0.23.0/src'
    19:53:23 DEBUG: make[1]: *** [check] Error 2
    19:53:23 DEBUG: make[1]: Leaving directory `/builddir/build/BUILD/mesos-0.23.0/src'
    19:53:23 DEBUG: make: *** [check-recursive] Error 1
    19:53:23 DEBUG: RPM build errors:
    ```


- Jie Yu


On May 21, 2015, 4:05 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 21, 2015, 4:05 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
>   include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
>   src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
>   src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
>   src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
>   src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
>   src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
>   src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
>   src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
>   src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review84857
-----------------------------------------------------------

Ship it!



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment136275>

    This looks like a good candidate for inlining too.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment136270>

    Dead code?



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment136271>

    Dead code?



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment136272>

    These two can easily and cleanly be inlined via lambdas.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137788>

    A good candidate for a C++11 lambda.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137789>

    Another candidate for a C++11 lamdba!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment136273>

    Why the body of this is not inlined should be commented here otherwise someone is likely wondering what was the reason for not inlining `_fetch` here.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137790>

    Why is os::chown no longer done any more? If we don't need to chown, why? And does that mean we can clean up this code considerably as the comment suggests?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137792>

    What about closing 'out' and 'err' here now?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137793>

    And need to close 'out' and 'err' here too.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment136274>

    This comment needs updating, looks like when you removed `__run` you forgot to do a global search for all instances of `__run`?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137794>

    CHECK_EQ(space, bytes);



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment137795>

    Is this really possibly anymore given the current design where 'adjust' will fail if the size we downloaded was bigger than what we first determined via 'fetchSize'?



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment137797>

    Please put '{' on newline and this could really use a comment!



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment137805>

    I'd like us to elaborate the comment here and add a TODO that we want a generic HTTP server for use in tests that has functionality like pausing requests. I'm not a huge fan of the half-actor half not strategy here, it's not something we want others to replicate and we should explicitly call that out so we don't get more broken windows.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment137803>

    This might break if someone has a machine configured where the default IP is not 127.0.0.1, but everything should stringify correctly for you anyway:
    
    return "http://" + stringify(self().address) + "/" + self().id + "/";



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment137806>

    Using locks here is dangerous because on a smaller machine depending on the number of requests we have come through a test we might actually deadlock the entire process. We should call this out explicitly and leave a TODO on how we can do this asynchronously.


- Benjamin Hindman


On May 21, 2015, 4:05 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 21, 2015, 4:05 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
>   include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
>   src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
>   src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
>   src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
>   src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
>   src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
>   src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
>   src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
>   src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 21, 2015, 9:05 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

memory::shared_ptr -> std::shared_ptr. This allows removing #include <stout/memory.hpp>, which blocked make distcheck.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
  include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
  src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
  src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
  src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
  src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
  src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
  src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
  src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
  src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 21, 2015, 5:06 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Fixed minor style issues.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
  include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
  src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
  src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
  src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
  src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
  src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
  src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
  src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
  src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 19, 2015, 8:58 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased to latest master.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f 
  include/mesos/type_utils.hpp 837be6f1844d5fa01c0fd84a585e7ff2cc0c987b 
  src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp ed4ee19c85387882ab2e31baa5610acb8e222d50 
  src/slave/containerizer/docker.cpp 408a4435a6f11973992486eac1659beeccc4beac 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 3e18617b0dbac58176bfd41dc550898eb6a4a79e 
  src/slave/containerizer/mesos/containerizer.cpp 696e359de66305512eedf8e269543fafa21f4bc3 
  src/slave/flags.hpp 5c57478fcfdbcbd8ac0e5c3c79809403054e96e6 
  src/slave/flags.cpp b5e25186dad36bc1306cc6ecb268aba951a18f7e 
  src/slave/slave.cpp 8e88482f41f37ce7f2559fe793565b66ac46fb35 
  src/tests/docker_containerizer_tests.cpp 154bf981c007ebcb8e0b2fe8551defb5ea2ba063 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp a60df75350beab8d7091cbe66213ecd920942fa4 
  src/tests/mesos.cpp 1d5639c85517229f3396b40f2d8bd421b2ed7325 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 15, 2015, 3:07 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Fixed remaining issues.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
  include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
  src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
  src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
  src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
  src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
  src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
  src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review83944
-----------------------------------------------------------



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment135003>

    AWAIT_READY, not AWAIT_READY_FOR, coz the default is 15 and that works fine, too.


- Bernd Mathiske


On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:07 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 13, 2015, 3:07 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Changed the way waiting for task completion is handled. Moved such waiting outside of launchTask(). Made helper functions create futures we can wait for with AWAIT_READY_FOR.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
  include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
  src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
  src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
  src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
  src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
  src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
  src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 12, 2015, 3:43 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Minor cleanups, mostly removing & from const assignments.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
  include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
  src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
  src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
  src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
  src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
  src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
  src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated May 4, 2015, 4:29 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed most open issues. The rest will be dealt with soon.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
  include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
  src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
  src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
  src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
  src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
  src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
  src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On May 1, 2015, 12:17 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 247
> > <https://reviews.apache.org/r/30774/diff/45/?file=947044#file947044line247>
> >
> >     Why are we doing 'driver->start();' here?

Good point, does not belong inside here. Moved it out of this method. Put extra calls at the call sites.


> On May 1, 2015, 12:17 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 681
> > <https://reviews.apache.org/r/30774/diff/45/?file=947044#file947044line681>
> >
> >     Capitlization please. Everywhere. ;-)

That would not be following the style guide: http://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Type_Names


> On May 1, 2015, 12:17 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 1179
> > <https://reviews.apache.org/r/30774/diff/45/?file=947044#file947044line1179>
> >
> >     Path
> >     {
> >       Try<bool> executable() const;
> >     };
> >     
> >     
> >     Path(runDirectory, commandFilename).executable();

Should we wait with this until Path has been fleshed out? This would be a separate review.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review82260
-----------------------------------------------------------


On April 30, 2015, 7:40 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated April 30, 2015, 7:40 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On May 1, 2015, 12:17 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 1179
> > <https://reviews.apache.org/r/30774/diff/45/?file=947044#file947044line1179>
> >
> >     Path
> >     {
> >       Try<bool> executable() const;
> >     };
> >     
> >     
> >     Path(runDirectory, commandFilename).executable();
> 
> Bernd Mathiske wrote:
>     Should we wait with this until Path has been fleshed out? This would be a separate review.

I added a TODO and this will lead to a separate JIRA ticket to enhance Path.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review82260
-----------------------------------------------------------


On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 13, 2015, 3:07 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review82260
-----------------------------------------------------------



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132957>

    s/Option<string>(/Some(/



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132958>

    s/c/create/



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132963>

    Try<MesosContainerizer*> create =
      MesosContainerizer::create(flags, true, fetcher);



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132959>

    Newline before comment please.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132961>

    Why are we doing 'driver->start();' here?



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132964>

    s/maxExpectedStatusUpdates/MAX_EXPECTED_STATUS_UPDATES/



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132966>

    auto await = [=](const TaskStatus& status) {
      CHECK_EQ(task.task_id(), status.task_id());
      if (status.state() == TASK_FINISHED) {
        return Nothing();
      }
      return taskStatusQueue.get()
        .then([=](const TaskStatus& status) {
          return await(status);
        });
    }
    
    Future<Nothing> finished = taskStatusQueue.get()
      .then([=](const TaskStatus& status) { return await(status); });
    
    AWAIT_READY_FOR(finished, Seconds(10));
    
    // ------------
    
    AWAIT_READY_FOR(awaitFinished(taskStatusQueue), Seconds(10));
    
    // ------------
    
    Promise<Nothing> finished;
    EXPECT_CALL(scheduler, statusUpdate(driver, _))
      .WillRepeatedly([=, &finished](
          const SchedulerDriver&, const TaskStatus& status) {
        CHECK_EQ(task.task_id(), status.task_id());
        if (status.state() == TASK_FINISHED) {
          finished.set(Nothing());
        }
      });
    
    AWAIT_READY_FOR(finished.future(), Seconds(10));
    
    // ------------------------
    
    But since we don't have C++11 lambdas as gmock actions yet, instead create your own action:
    
    ACTION_TEMPLATE(FinishedStatusUpdate, promise)
    {
      const TaskStatus& status = arg1;
      // TODO: check
      if (status.state() == TASK_FINISHED) {
        promise->set(Nothing());
      }
    }
    
    And finally, let's pull this out of s/runTask/launchTask/ so it has consistent behavior with launchTasks.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132968>

    InvokeLambda<N>(args, lambda)
    {
      return lambda(get<0>(args), get<1>)(args), get<2>(args), ...);
    }
    
    InvokeVoidLambda<N>(args, lambda)
    {
      lambda(get<0>(args), get<1>)(args), get<2>(args), ...);
    }



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132980>

    Let's rename this to something that is more indicative of what's happening when we reach or block on a FetcherProcess::_fetch.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132981>

    CHECK_READY
    
    Here and everywhere else please. ;-)



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132982>

    const&



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132986>

    Capitlization please. Everywhere. ;-)



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132987>

    if (strings::contains(event.request->path, COMMAND_NAME)) {
    
    }
    
    Below please too!



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132989>

    Unroll please, the switch is harder to grok below.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment132988>

    Path
    {
      Try<bool> executable() const;
    };
    
    Path(runDirectory, commandFilename).executable();



src/tests/fetcher_tests.cpp
<https://reviews.apache.org/r/30774/#comment132990>

    EXPECT_ERROR
    
    (Search for 'isError())' and replace. ;-) )



src/tests/fetcher_tests.cpp
<https://reviews.apache.org/r/30774/#comment132991>

    s/..../bernd/



src/tests/fetcher_tests.cpp
<https://reviews.apache.org/r/30774/#comment132992>

    EXPECT_TRUE(strings::contains(fetch.failure(), "chown"));
    
    (How about we just search and replace .find?)



src/tests/mesos.cpp
<https://reviews.apache.org/r/30774/#comment132993>

    Let's double check whether or not this is really necessary?


- Benjamin Hindman


On May 1, 2015, 2:40 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated May 1, 2015, 2:40 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
>   include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
>   src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
>   src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
>   src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
>   src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
>   src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
>   src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
>   src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
>   src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated April 30, 2015, 7:40 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Fixed most issues, but not all. Let's talk about theremaining ones before proceeding!


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
  include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
  src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
  src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
  src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
  src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
  src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
  src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated April 29, 2015, 1:42 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f 
  include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb 
  src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f 
  src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 
  src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 
  src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 
  src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d 
  src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated April 13, 2015, 5:45 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed the latest 2 issues.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 3a8e8bf303e0576c212951f6028af77e54d93537 
  include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
  src/Makefile.am d15a37365bcdd5c3906160b46b389635b38b1673 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp 6893684e6d199a5d69fc8bba8e60c4acaae9c3c9 
  src/slave/containerizer/docker.cpp f9fb07806e3b7d7d2afc1be3b8756eac23b32dcd 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp e4136095fca55637864f495098189ab3ad8d8fe7 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp 35f56252cfda5011d21aa188f33cc3e68a694968 
  src/slave/slave.cpp a0595f93ce4720f5b9926326d01210460ccb0667 
  src/tests/docker_containerizer_tests.cpp c772d4c836de18b0e87636cb42200356d24ec73d 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 42e42ac425a448fcc5e93db1cef1112cbf5e67c4 
  src/tests/mesos.cpp fc534e9febed1e293076e00e0f5c3879a78df90f 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated April 10, 2015, 4:33 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 3a8e8bf303e0576c212951f6028af77e54d93537 
  include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
  src/Makefile.am fa609da08e23d6595a3f6d2efddd3e333b6c78f1 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp 6893684e6d199a5d69fc8bba8e60c4acaae9c3c9 
  src/slave/containerizer/docker.cpp f9fb07806e3b7d7d2afc1be3b8756eac23b32dcd 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp e4136095fca55637864f495098189ab3ad8d8fe7 
  src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 
  src/slave/flags.cpp 35f56252cfda5011d21aa188f33cc3e68a694968 
  src/slave/slave.cpp 9fec023b643d410f4d511fa6f80e9835bab95b7e 
  src/tests/docker_containerizer_tests.cpp c772d4c836de18b0e87636cb42200356d24ec73d 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 0e98572a62ae05437bd2bc800c370ad1a0c43751 
  src/tests/mesos.cpp 02cbb4b8cf1206d0f32d160addc91d7e0f1ab28b 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 24, 2015, 6:57 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

New structure for the core algorithm that attacks invariants more locally. Thanks to Ben H for advising and trail blazing how to do this!

Todo: remove contentionBarrier() and mock _fetch() instead, rebase.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
  include/mesos/type_utils.hpp cdf5864389a72002b538c263d70bcade2bdffa45 
  src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
  src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
  src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 21, 2015, 11:54 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Cleanups. Thanks, Till!


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
  src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
  src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
  src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 19, 2015, 10:48 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Fixed 3 more review issues.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
  src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
  src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
  src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 18, 2015, 11:48 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 726
> > <https://reviews.apache.org/r/30774/diff/37/?file=897704#file897704line726>
> >
> >     I'm not sure I understand, the error is never logged and in the end we simply return 0 if os::find returns a error. To me that looks like we're ignoring if Try has an error right?

No problem, I'll rewrite it and put a comment: When there is an error then the cache directory does not exist, which means the number of files in the cache is zero.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77025
-----------------------------------------------------------


On March 18, 2015, 11:43 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:43 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review77025
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment124810>

    I'm not sure I understand, the error is never logged and in the end we simply return 0 if os::find returns a error. To me that looks like we're ignoring if Try has an error right?


- Timothy Chen


On March 19, 2015, 6:43 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 19, 2015, 6:43 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
>   src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
>   src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
>   src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 18, 2015, 11:43 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed most of Tim's review, left comments for the rest. Tim, Ben, many thanks for the reviews! Please revisit your issues and update or close the open ones, given my reponses.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
  src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
  src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
  src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 17, 2015, 6:59 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased to current master.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md 7119b1421ac1506fa118e9f91d07e027dec3d92e 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto ec8efaec13f54a56d82411f6cdbdb8ad8b103748 
  src/Makefile.am 7a06c7028eca8164b1f5fdea6a7ecd37ee6826bb 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
  src/slave/flags.hpp dbaf5f532d0bc65a6d16856b8ffcc2c06a98f1fa 
  src/slave/slave.cpp 0f99e4efb8fa2b96f120a3e49191158ca0364c06 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 17, 2015, 4:52 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Added more rational for the code structure in fetch() to docs/fetcher-cache-internals. Refactored fetch() somewhat: replaced cleanupNewCacheEntries() with ___fetch(), inlined what was in makeCacheItems() and applyFallbacks(). There should now be a more straightforward  "flow" to the whole process. However, I did not find a suitable substitute for FetcherItem as phase 1 future parameter. Sticking with that.

(I know there is a slight regression wrt. os::stat::size, so this is not rebased. Will fix that in the next patch. Apart from this, this patch is fully functional and runs all tests on an older version of master.)


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md fc3afec248b534b1d5eb625eb66de5f90cd8cd33 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 11, 2015, 10:50 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md fc3afec248b534b1d5eb625eb66de5f90cd8cd33 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am 3059818231c46484039d179cd6916932eff6cd68 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp 71ae84bbfcef208cc2ee603f3c8a79225e48a7d5 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp 45e35204d1aa876fa0c871acf0f21afcd5ababe8 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 11, 2015, 5:47 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed almost all issues. Simplified fetcher cache test source code.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/configuration.md fc3afec248b534b1d5eb625eb66de5f90cd8cd33 
  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 7, 2015, 7:21 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed Tim's latest review.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/launcher/fetcher.cpp, line 178
> > <https://reviews.apache.org/r/30774/diff/32/?file=887354#file887354line178>
> >
> >     You log the extraction command but in this case don't log the copy command.
> >     
> >     I think to be consistent, let's not log the command, and like you do here only log when the command fails.
> >     
> >     What you think?

Logging the command in both cases now.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.hpp, line 195
> > <https://reviews.apache.org/r/30774/diff/32/?file=887358#file887358line195>
> >
> >     Why not just store the Path and return that?

"directory" is a temporary artefact that will disappear once we refactor so that flags gets injected into the fetcher. I added a comment syaing that.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 175
> > <https://reviews.apache.org/r/30774/diff/32/?file=887359#file887359line175>
> >
> >     Let's use strings::contains instead of find to be consistent here.

Also fixed all other occurences.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/slave.cpp, line 3710
> > <https://reviews.apache.org/r/30774/diff/32/?file=887363#file887363line3710>
> >
> >     Why is this just a Failure but the other recover is a LOG(FATAL)? Shouldn't we exit here too if unable to recover cache is a critical event?

The method we are in returns a future, so we can return a Failure here. This leads to exiting ventually. At the other site, the method we are in only returns void. Suggestions for that?


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75491
-----------------------------------------------------------


On March 6, 2015, 5:46 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 6, 2015, 5:46 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > include/mesos/fetcher/fetcher.proto, line 58
> > <https://reviews.apache.org/r/30774/diff/32/?file=887350#file887350line58>
> >
> >     It's harder to make a optional field required, but it's much easier the other way around.
> >     
> >     If we always want it to be required, I think we should make the sandbox a required field.
> 
> Bernd Mathiske wrote:
>     There was some discussion about whether this field should be required or not. The general idea here is that a task might be able to run without fetching anything into its sandbox. In this case, the framework may get away without naming the sandbox. But since a task always has one, we could also make it required. I am impartial in this choice, but I see that your argument that required->optional is easier has pull.

I have heard good arguments both ways. Here is how I see it. 

For the recipient of a message, "optional" is the preferred choice. Then any legacy recipient's code is always prepared for everything and robust wrt. changing to "required". Not the other way around.

But for the sender, "required" is the better choice, making sender code more robust. If legacy senders still provide the field when it has become optional, that's OK. Not the other way around.

So which side are we on in this case? As much as this is an internal protocol, we are on neither side and we can change this in arbitrary ways. 

This is an external protocol if someone else than a Mesos slave uses mesos-fetcher. (Maybe a special external containerizer.) Then we are providing the message recipient and we have to be on that side. Therefore I am voting for "optional".


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75491
-----------------------------------------------------------


On March 7, 2015, 7:21 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 7, 2015, 7:21 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > include/mesos/fetcher/fetcher.proto, line 58
> > <https://reviews.apache.org/r/30774/diff/32/?file=887350#file887350line58>
> >
> >     It's harder to make a optional field required, but it's much easier the other way around.
> >     
> >     If we always want it to be required, I think we should make the sandbox a required field.

There was some discussion about whether this field should be required or not. The general idea here is that a task might be able to run without fetching anything into its sandbox. In this case, the framework may get away without naming the sandbox. But since a task always has one, we could also make it required. I am impartial in this choice, but I see that your argument that required->optional is easier has pull.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/hdfs/hdfs.hpp, line 82
> > <https://reviews.apache.org/r/30774/diff/32/?file=887353#file887353line82>
> >
> >     IMO this should be in another patch and we can get this commited right away.

I WAS in another patch: 30616. BenH advised to put all fetcher cache related patches that are not for stout or libprocess together in one patch.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.hpp, line 76
> > <https://reviews.apache.org/r/30774/diff/32/?file=887358#file887358line76>
> >
> >     Are these static methods going to be used somewhere else? Quite a lot of static methods now in the header and perhaps we just need to put the implementation in the cpp file, BenH also mentioned this last time in the review meeting.

Yes, these are all used both in launcher/fetcher.cpp as well as containerizer/fetcher.cpp. They are factored out for consistency and easier maintenance. In previous fetcher implementations they were not, enjoying a duplicitous existence.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.hpp, line 191
> > <https://reviews.apache.org/r/30774/diff/32/?file=887358#file887358line191>
> >
> >     I think there is a bit of complication with this interface, where a user has to call two things at the end of a downloading from an Entry:
> >     
> >     - unreference
> >     - complete/fail
> >     
> >     And I don't see how one ever wants to use them without each other.
> >     
> >     Why not hide unreference all together, and decrement the reference count in complete or fail?
> >     
> >     This way it's a lot less error prone, and harder to make mistakes with future changes.

unreference is not only used for new cache entries that find closure in completion or failure. It is primarily used for pre-existing entries that have downloads by concurrent fetch runs. In both cases, we need to call unreference at the very end of fetching in our current run. If we hid unreference in only one of the two cases, we'd have a bug.

How can anything be less error-prone than calling unreference on everything that got referenced? By hard-coupling "reference" with storing the unreference action in a collection. I had a version exactly like this but it was turned down by a reviewer, because it introduced an extra class.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 749
> > <https://reviews.apache.org/r/30774/diff/32/?file=887359#file887359line749>
> >
> >     What's the point of this empty branch?

I like putting comments about what happens in the code path / case not taken in an empty branch instead of placing them in a less directly related place. This is much more clear IMHO.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 1087
> > <https://reviews.apache.org/r/30774/diff/32/?file=887359#file887359line1087>
> >
> >     This seems to be very easy to make mistakes with setSpace, especially when it's not even expected to be called more than once? 
> >     
> >     I thought you said you're going to change it so it's only set once during initialization?

BenH and I decided to postpone this refactoring. It would lead to a lot of additional code changes for ni current semantic or general architecture changes.


> On March 6, 2015, 2:15 p.m., Timothy Chen wrote:
> > src/tests/mesos.hpp, line 703
> > <https://reviews.apache.org/r/30774/diff/32/?file=887367#file887367line703>
> >
> >     We want to have a consistent naming style across all the tests and files, and we usually in all the tests in Mesos, the "unmocked" methods tend to just have a prefix of "_", so run -> _run

If you use "_run" how do you distinguish this from a continuation of the same name? We cannot possibly use this naming scheme. Please either convert to mine or come up with a better one. I think that unmocked-something makes it very clear what is going on without making first readers guess or having to put extra comments. So I'd prefer leaving it like that.

(I had mocked a method _fetch in an earlier patch and there is also a continuation __fetch...)


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75491
-----------------------------------------------------------


On March 6, 2015, 5:46 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 6, 2015, 5:46 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review75491
-----------------------------------------------------------



include/mesos/fetcher/fetcher.proto
<https://reviews.apache.org/r/30774/#comment122581>

    It's harder to make a optional field required, but it's much easier the other way around.
    
    If we always want it to be required, I think we should make the sandbox a required field.



src/hdfs/hdfs.hpp
<https://reviews.apache.org/r/30774/#comment122582>

    IMO this should be in another patch and we can get this commited right away.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122584>

    Why not log the error if it's there? available.error()



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122585>

    You log the extraction command but in this case don't log the copy command.
    
    I think to be consistent, let's not log the command, and like you do here only log when the command fails.
    
    What you think?



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122591>

    Failed to chmod executable



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122597>

    CHECK_EQ(item.action(), FetcherInfo::Item::DOWNLOAD_AND_CACHE)



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment122598>

    Are these static methods going to be used somewhere else? Quite a lot of static methods now in the header and perhaps we just need to put the implementation in the cpp file, BenH also mentioned this last time in the review meeting.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment122601>

    I think there is a bit of complication with this interface, where a user has to call two things at the end of a downloading from an Entry:
    
    - unreference
    - complete/fail
    
    And I don't see how one ever wants to use them without each other.
    
    Why not hide unreference all together, and decrement the reference count in complete or fail?
    
    This way it's a lot less error prone, and harder to make mistakes with future changes.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment122732>

    Why not just store the Path and return that?



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment122731>

    Where is this being used?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122588>

    Let's use strings::contains instead of find to be consistent here.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122736>

    What's the point of this empty branch?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122738>

    This seems to be very easy to make mistakes with setSpace, especially when it's not even expected to be called more than once? 
    
    I thought you said you're going to change it so it's only set once during initialization?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122733>

    We should be more explicit here, that we're claiming more than the space we've set to.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122734>

    Add some message?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment122735>

    Add some message?



src/slave/slave.cpp
<https://reviews.apache.org/r/30774/#comment122730>

    Why is this just a Failure but the other recover is a LOG(FATAL)? Shouldn't we exit here too if unable to recover cache is a critical event?



src/tests/mesos.hpp
<https://reviews.apache.org/r/30774/#comment122578>

    We want to have a consistent naming style across all the tests and files, and we usually in all the tests in Mesos, the "unmocked" methods tend to just have a prefix of "_", so run -> _run


- Timothy Chen


On March 6, 2015, 1:46 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 6, 2015, 1:46 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher-cache-internals.md PRE-CREATION 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
>   src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
>   src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 6, 2015, 5:46 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Added outlook on future features and made other small improvements to documentation. Based on feedback from Robert Lacroix and Jay Buffington.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 5, 2015, 3:15 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Removed Fetcher::clearCache().


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 4, 2015, 4:54 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Refactored fetch() and its continuations. Now separating cached and uncached URIs at the beginning. Simplified a lot of control flow after removing async file operations. All tests run.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 3, 2015, 11:40 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Removed a major piece of both functionality and complexity: no more async() for file operations. This is deemed good enough for MVP. The legacy fetcher behavior is not made worse by this.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 3, 2015, 9:53 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Removed binary files.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 3, 2015, 9:30 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Patched in suggested patches from Benh. Thanks, Ben!


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  docs/images/fetch_cache.jpg PRE-CREATION 
  docs/images/fetch_components.jpg PRE-CREATION 
  docs/images/fetch_flow.jpg PRE-CREATION 
  docs/images/fetch_force1.jpg PRE-CREATION 
  docs/images/fetch_force2.jpg PRE-CREATION 
  docs/images/fetch_state.jpg PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 3, 2015, 5:01 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

More fetcher cache internals documentation with diagrams.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  docs/images/fetch_cache.jpg PRE-CREATION 
  docs/images/fetch_components.jpg PRE-CREATION 
  docs/images/fetch_flow.jpg PRE-CREATION 
  docs/images/fetch_force1.jpg PRE-CREATION 
  docs/images/fetch_force2.jpg PRE-CREATION 
  docs/images/fetch_state.jpg PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 2, 2015, 10:27 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Fetcher cache internals doc with overview diagram, control flow diagram, cache entry state diagram.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher-cache-internals.md PRE-CREATION 
  docs/fetcher.md PRE-CREATION 
  docs/images/fetch_components.jpg PRE-CREATION 
  docs/images/fetch_flow.jpg PRE-CREATION 
  docs/images/fetch_state.jpg PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 1, 2015, 12:27 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Added a test for bypassing the cache when upfront size fetching fails.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 1, 2015, 7:42 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased to current master, changed nothing in the outcome, just adjusted the diff files.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am d299f07d865080676ca8a550cf6005c6ab32839f 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp a06d68032f26ccb3f786b6ea7c3a6c3c52449bd2 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e 
  src/tests/mesos.hpp e91e5e484eea4587ac8f2eb9cefeab4acc9f4615 
  src/tests/mesos.cpp c8f43d21b214e75eaac2870cbdf4f03fd18707d1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74695
-----------------------------------------------------------


Bad patch!

Reviews applied: [30606, 30609, 30774]

Failed command: ./support/apply-review.sh -n -r 30774

Error:
 2015-03-01 10:54:39 URL:https://reviews.apache.org/r/30774/diff/raw/ [169279/169279] -> "30774.patch" [1]
error: patch failed: src/tests/fetcher_tests.cpp:48
error: src/tests/fetcher_tests.cpp: patch does not apply
error: patch failed: src/tests/mesos.cpp:47
error: src/tests/mesos.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On March 1, 2015, 10:35 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated March 1, 2015, 10:35 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 0ae0a10b41a2c0f7459c771b31c76bbc0c02df4f 
>   src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
>   src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated March 1, 2015, 2:35 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Improved fetcher cache tests, introduced two cache eviction tests. One tests whether eviction succeeds, the other also tests what happens if it fails.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 0ae0a10b41a2c0f7459c771b31c76bbc0c02df4f 
  src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
  src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 26, 2015, 7:58 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased to current master. Fixed a bug in fetching from local files.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 0ae0a10b41a2c0f7459c771b31c76bbc0c02df4f 
  src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
  src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74271
-----------------------------------------------------------



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120814>

    Ah, I found this now. Was looking at the wrong revision. I am programming defensively here. There is no reason to yell error just because a file we want to delete does not exist. os::rm() would complain. How could the file disappear? Maybe by a human operator wiping the cache from the outside. The fetcher cache should be able to survive this!


- Bernd Mathiske


On Feb. 26, 2015, 6:03 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 26, 2015, 6:03 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
>   src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp fed0105946da579a38357a30e7ae56e646e05b89 
>   src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
>   src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
>   src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
>   src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
>   src/slave/containerizer/fetcher.cpp d290f95251def3952c5ee34f600e1d71467f6293 
>   src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
>   src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
>   src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
>   src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
>   src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp 2438620e2db94c3c56336fa5d8e69a18fe8f3bac 
>   src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
>   src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 26, 2015, 6:03 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased to current master. Addressed all issues except 1. Removed Hdfs fetch test, which will be reinstated by MESOS-2390, which is reviewable.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 9df972d750ce1e4a81d2e96cc508d6f83cad2fc8 
  src/Makefile.am 17d0d7aa7361c3a373f6863d36b0a4767f5c05c4 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp fed0105946da579a38357a30e7ae56e646e05b89 
  src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec 
  src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 
  src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 
  src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 
  src/slave/containerizer/fetcher.cpp d290f95251def3952c5ee34f600e1d71467f6293 
  src/slave/containerizer/mesos/containerizer.hpp ae61a0fcd19f2ba808624312401f020121baf5d4 
  src/slave/containerizer/mesos/containerizer.cpp ec4626f903d44c0911093ff763ef16ad27c418a9 
  src/slave/flags.hpp 56b25caf3901b38bdecb50310e8bcae0b114efa8 
  src/slave/slave.cpp 9f31fa46304398e8f87b41b55d8f4cfd4aba10b9 
  src/tests/docker_containerizer_tests.cpp 06cd3d89ecbaaac17ae6970604b21fbe29f6e887 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp 2438620e2db94c3c56336fa5d8e69a18fe8f3bac 
  src/tests/mesos.hpp f7a0d057edea1a7ec7ae3bb9bc729230bf7dd46d 
  src/tests/mesos.cpp 23f790cbb289f6483dcdfa6ecccd462360ce02f1 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 165
> > <https://reviews.apache.org/r/30774/diff/18/?file=875537#file875537line165>
> >
> >     Doesn't cache items have TTLs? Then frameworks should never get into content that is stale for a long time?

TTLs are not implemented. (Regardless I would still give the same advice if they were.)


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 212
> > <https://reviews.apache.org/r/30774/diff/18/?file=875537#file875537line212>
> >
> >     What's the policy for cache eviction. Is this configurable?

It is intentionally unspecified for users so we can change it.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 183
> > <https://reviews.apache.org/r/30774/diff/18/?file=875537#file875537line183>
> >
> >     Probably worth mentioning you're only considering the size of the symlink.

Turns out I do want to follow links after all.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 195
> > <https://reviews.apache.org/r/30774/diff/18/?file=875537#file875537line195>
> >
> >     Can you provide a quick way to verify if the URI content size can be queried up front? And this only applies to remote URIs right?
> >     
> >     Since you provided a big warning, I think it's best to also tell the users how to quickly verify this instead of going around and figuring out what it means to verify.

I state a couple of paragrahps above exactly how sizes are determined. This applies to all supported URIs except ftp.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > docs/fetcher.md, line 229
> > <https://reviews.apache.org/r/30774/diff/18/?file=875537#file875537line229>
> >
> >     somehwere -> somewhere.
> >     And if you're going to put the default value here, why not put the actual directory name instead of  "somewhere in the work dir"?
> >     
> >     I would also suggest removing the defaults as the flags should tell the users and we don't want to keep two sources of truth.

If we don't have any defaults, we cannot run tests. "somewhere in the work dir" is important info, because it clarifies that tests will run, but it is not ideal for production. I will be vague about the cache size, too, though, to remove one place of truth.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > src/launcher/fetcher.cpp, line 80
> > <https://reviews.apache.org/r/30774/diff/18/?file=875542#file875542line80>
> >
> >     I don't think this is normal LOG(INFO), we usually log actual commands with VLOG(1).

I have never seen VLOG in launcher/fetcher.cpp, only LOG(INFO).


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > src/launcher/fetcher.cpp, line 136
> > <https://reviews.apache.org/r/30774/diff/18/?file=875542#file875542line136>
> >
> >     Why not put both log in one line? They might get seperated out by other log output if you don't concat them.

We need only one of these.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > src/launcher/fetcher.cpp, line 443
> > <https://reviews.apache.org/r/30774/diff/18/?file=875542#file875542line443>
> >
> >     I don't know if it makes sense to log this as INFO.
> >     INFO is the default and it shows up on every task's stdout. What's your rationale doing INFO here?

I believe it goes into stderr. If we get a call that the fetcher cache is broken somewhere, we can inspect with what parameters mesos-fetcher has been called for a specific task, without rerunning it.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 330
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line330>
> >
> >     Why set space every time we fetch? Shouldn't this gets called just once?
> >     
> >     And I remember you mentioned setting zero space will disable the cache right? Where is that happening?

This is an artefact of flags not being injected. This will be refactored in a later ticket. Putting a TODO comment.

fetcher_cache_size is a slave cmd line flag.


> On Feb. 25, 2015, 10:06 a.m., Timothy Chen wrote:
> > src/slave/containerizer/fetcher.cpp, line 1138
> > <https://reviews.apache.org/r/30774/diff/18/?file=875547#file875547line1138>
> >
> >     If we don't want people to ever change the size, why not put this in a one time initialization method?

Let's correct this later. It will also change once I introduce eviction tests, which require dynamic cache size updates.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74040
-----------------------------------------------------------


On Feb. 25, 2015, 9:54 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 9:54 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review74040
-----------------------------------------------------------



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120509>

    Doesn't cache items have TTLs? Then frameworks should never get into content that is stale for a long time?



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120510>

    Probably worth mentioning you're only considering the size of the symlink.



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120516>

    Can you provide a quick way to verify if the URI content size can be queried up front? And this only applies to remote URIs right?
    
    Since you provided a big warning, I think it's best to also tell the users how to quickly verify this instead of going around and figuring out what it means to verify.



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120518>

    What's the policy for cache eviction. Is this configurable?



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120521>

    somehwere -> somewhere.
    And if you're going to put the default value here, why not put the actual directory name instead of  "somewhere in the work dir"?
    
    I would also suggest removing the defaults as the flags should tell the users and we don't want to keep two sources of truth.



docs/fetcher.md
<https://reviews.apache.org/r/30774/#comment120523>

    How about just mention you can also disable the fetcher cache by specifying zero bytes in the cache size flag?



src/hdfs/hdfs.hpp
<https://reviews.apache.org/r/30774/#comment120524>

    Remove the extra space in the beginning



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120525>

    I don't think this is normal LOG(INFO), we usually log actual commands with VLOG(1).



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120526>

    Why not put both log in one line? They might get seperated out by other log output if you don't concat them.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120527>

    I don't know if it makes sense to log this as INFO.
    INFO is the default and it shows up on every task's stdout. What's your rationale doing INFO here?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120529>

    Why set space every time we fetch? Shouldn't this gets called just once?
    
    And I remember you mentioned setting zero space will disable the cache right? Where is that happening?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment120528>

    If we don't want people to ever change the size, why not put this in a one time initialization method?


- Timothy Chen


On Feb. 25, 2015, 5:54 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 5:54 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 25, 2015, 9:54 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Refactored cache eviction. The only remaining planned changes are: 1. Add eviction tests, 2. rebase.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 25, 2015, 2:19 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed Tim's review.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review73553
-----------------------------------------------------------


Bad patch!

Reviews applied: [30609, 30606, 30774]

Failed command: ./support/apply-review.sh -n -r 30774

Error:
 2015-02-23 12:45:57 URL:https://reviews.apache.org/r/30774/diff/raw/ [159862/159862] -> "30774.patch" [1]
error: patch failed: src/Makefile.am:1301
error: src/Makefile.am: patch does not apply
error: patch failed: src/launcher/fetcher.cpp:34
error: src/launcher/fetcher.cpp: patch does not apply
error: patch failed: src/slave/containerizer/fetcher.hpp:20
error: src/slave/containerizer/fetcher.hpp: patch does not apply
error: patch failed: src/slave/containerizer/fetcher.cpp:16
error: src/slave/containerizer/fetcher.cpp: patch does not apply
error: patch failed: src/tests/fetcher_tests.cpp:56
error: src/tests/fetcher_tests.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Feb. 23, 2015, 11:33 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 23, 2015, 11:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
>   src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
>   src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
>   src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
>   src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
>   src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 23, 2015, 3:33 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Removed prefetching so there is a bit less to review.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 22, 2015, 10:09 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Added another prefetching test.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 22, 2015, 12:39 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Another test, another bug fix.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 21, 2015, 5:26 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

More tests (with more mock methods), more bug fixes, more VLOG statements.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 19, 2015, 7:14 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Removed attempt to get #fetched files from mesos-fetcher process status; this does not work unless we rewrite Subprocess somewhat. More documentation. More tests, also negative ones. Found some bugs with these and fixed them.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 18, 2015, 4:43 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Complete description of the fetcher and the fetcher cache in "docs/fetcher.md". Read this first!


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 18, 2015, 12:28 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rebased.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 507845c493f65e154214fc7e562206e452990469 
  src/Makefile.am d372404d84c9ac0bc49da7407ad366701b9586a6 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 12d6e92b814076fd423a7738e030dc6ce6954761 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp 074a2d82dcd882e52f8cd62ed7493295596acb26 
  src/slave/containerizer/mesos/containerizer.cpp d5b90d12d63becfeb4c3efa9c6f5d826417f582c 
  src/slave/flags.hpp ddb32593566b71bcf0b66adeeb889e43ef911084 
  src/slave/slave.cpp ec7ec1356e745bb07484ae1755c9183b038043b3 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 
  src/tests/mesos.cpp d76ed8c837da162d782de84e935fe8e11e01d6b0 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 18, 2015, 12:07 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Added MesosTest::SetUp and made it protected. Now using MesosTest::Setup and TearDown in fetcher cache tests. Test touchups and more comments.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 17, 2015, 8:40 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Added some features for MESOS-2073: forced cache updating, prefetching. Added fetcher documentation file in "docs", to be expanded.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2073
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  docs/fetcher.md PRE-CREATION 
  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 17, 2015, 3:02 a.m., Alexander Rukletsov wrote:
> > src/tests/fetcher_cache_tests.cpp, line 117
> > <https://reviews.apache.org/r/30774/diff/8/?file=866273#file866273line117>
> >
> >     Don't you need to explicitly call `MesosTest::SetUp()`?

Nope.


> On Feb. 17, 2015, 3:02 a.m., Alexander Rukletsov wrote:
> > src/tests/fetcher_cache_tests.cpp, line 165
> > <https://reviews.apache.org/r/30774/diff/8/?file=866273#file866273line165>
> >
> >     Same here, don't you need to explicitly call `MesosTest::TearDown()` to make sure things are cleaned up?

Nope.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review72694
-----------------------------------------------------------


On Feb. 17, 2015, 8:40 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 8:40 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 17, 2015, 3:02 a.m., Alexander Rukletsov wrote:
> > src/tests/fetcher_cache_tests.cpp, line 117
> > <https://reviews.apache.org/r/30774/diff/8/?file=866273#file866273line117>
> >
> >     Don't you need to explicitly call `MesosTest::SetUp()`?
> 
> Bernd Mathiske wrote:
>     Nope.

Revisited. I inferred incorrectly from this never being used elsewhere and from the fact that MesosTest::SetUp() is private that it should not be used by someone's intention. I am calling this wrong now and will make MesosTest::SetUp() protected in my next patch, then call it.


> On Feb. 17, 2015, 3:02 a.m., Alexander Rukletsov wrote:
> > src/tests/fetcher_cache_tests.cpp, line 165
> > <https://reviews.apache.org/r/30774/diff/8/?file=866273#file866273line165>
> >
> >     Same here, don't you need to explicitly call `MesosTest::TearDown()` to make sure things are cleaned up?
> 
> Bernd Mathiske wrote:
>     Nope.

This failed without the impossible-to-call MesosTest::SetUp(). Will call it from now on.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review72694
-----------------------------------------------------------


On Feb. 17, 2015, 8:40 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 8:40 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2073
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   docs/fetcher.md PRE-CREATION 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Alexander Rukletsov <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review72694
-----------------------------------------------------------



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment118763>

    Don't you need to explicitly call `MesosTest::SetUp()`?



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment118764>

    Same here, don't you need to explicitly call `MesosTest::TearDown()` to make sure things are cleaned up?


- Alexander Rukletsov


On Feb. 17, 2015, 9:35 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 9:35 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 17, 2015, 1:35 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Put code into Setup() and TearDown() as much as possible. Now testing in HTTP tests whether fetch contention occurs, i.e. whether FetcherProcess::_fetch() gets called for all tasks before they can proceed with downloading anything. By implication this exercises the code that sorts out what to do with each URI and determines that some of them need to simply wait for another task to download what they requested. This is further confirmed by checking the number of requests to the HTTP server, which are now differentiated by URI.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 15, 2015, 7:28 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

In fetcher cache tests: working towards Setup/Teardown to include more code, removed checkpointing mode distinction.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 15, 2015, 2:29 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Now using process::Queue to wait for task status updates in fetcher cache tests.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 14, 2015, 10:34 p.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Shorter description that could be used as commit message, offloaded extra detail to the testing section.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description (updated)
-------

Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.


Diffs
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing (updated)
-------

make check

--- longer Description: ---

-Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 14, 2015, 3 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Extended description.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description (updated)
-------

Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:

30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.

30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)

30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.

30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 

30039: Enables fetcher cache actions in the mesos fetcher program.

30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 

30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.

30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.

30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.

30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).

30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.

30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
- mesos-fetcher does not run until evictions have been successful
- Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
- Reservations can be partly from available space, partly from evictions. All math included :-)
- To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
- Size-determination requests are now synchronized, too. Only one per URI in play happens.
- There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
- Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.


Diffs
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 14, 2015, 2:38 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Fixed minor formatting issues.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 14, 2015, 1:41 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Rewrite of fetcher.hpp and fetcher.cpp. Now individual code paths per cache file leading to a single invocation of mesos-fetcher. More error situations covered (trying to cover all known/possible ones, actually).


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 10, 2015, 1:14 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Addressed most issues. Only left those that require refactoring for later.


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description
-------

Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.


Diffs (updated)
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing
-------

make check


Thanks,

Bernd Mathiske


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, lines 132-135
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line132>
> >
> >     There is only "checkpointing" mode now, so don't bother trying to test both cases.
> 
> Bernd Mathiske wrote:
>     I tried removing this distinction and tests failed, both the recovery test and the others that don't test it. Keeping the distinction for now.

Found a bug, works now without distinction.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71575
-----------------------------------------------------------


On Feb. 15, 2015, 2:29 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 15, 2015, 2:29 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, lines 332-334
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line332>
> >
> >     We've got a concurrent queue in libprocess called 'Queue'. See scheduler_tests.cpp for how it's used. I'd love to see the 'Enqueue' gmock action be made more generic so that you can use it here too (I'm confident it can, and can help if need be, and would rather use that then introduce another concurrent queuing like data structure, even if it's only for the tests).

Now using Queue.

Let's discuss and elaborate on the Enqueue part later.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71575
-----------------------------------------------------------


On Feb. 14, 2015, 10:34 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 14, 2015, 10:34 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 438
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line438>
> >
> >     There is a ton of state, expectations, control flow requirements, etc, throughout these tests. This makes them very hard to follow, and thus hard to understand if you're really testing the things you want to test (i.e., concurrent requests, etc). While it's probably more code, I'd rather have seen each test case testing exactly what you wanted to test rather than a bunch of functions pulled out to cover "commonality". It makes it very difficult for me to determine if certain test situations are covered!
> >     
> >     For example, in this function you await tasks if in "concurrent" mode. Yet, I'm pretty sure tasks aren't going to run in parallel unless there are enough resources to do so, and tracking down the size of the tasks and how many resources were launched with the slave is so non-local that it requires looking all over this source file and after doing so I'm not convinced that anything can actually run in parallel since the default slave has 2 CPUs and each task takes 1 CPU (so maybe 2 tasks run in parallel but not all of them?).
> >     
> >     Why not be more explicit in the tests, even if it's more code, so that we're convinced we're testing the things we want to test? My guess is that there will still be things we can pull out and abstract after having done that.
> 
> Bernd Mathiske wrote:
>

Indeed, we are not testing everything that needs to be tested yet. More tests will be added. 

Each test does state in a comment atop the test method what it is exactly that it tests. I have removed some the commonality functions and put their content into Setup() and TearDown(). What remains is "executeTasks()" and "await<SuchAndSuch>()". Inlining these would not make anything more readable IMHO.

I think that in the latest update I have now established that the code in _fetch() that is responsible for sorting out concurrent fetching is well-exercised. (See RR update message and new comments in the code.) Hardware parallelism is not relevant here, nor is that any tasks execute in parallel. In fact, I am trying hard to let no task run at all before fetching has become concurrent.

There are always enough resources to run "concurrently" in the sense needed here. See Setup() and CommandInfo usage. We never use them up.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71575
-----------------------------------------------------------


On Feb. 17, 2015, 1:35 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2015, 1:35 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> --- longer Description: ---
> 
> -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order:
> 
> 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway.
> 
> 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.)
> 
> 30036: Extends the CommandInfo::URI protobuf with a boolean "caching" field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed.
> 
> 30037: Extends the fetcher info protobuf with "actions" (fetch directly bypassing the cache, fetch through the cache, retrieve from the cache). Switches the basis for dealing with uris to "items", which contain the uri, the action, and potentially a cache file name. Refactors fetch() and run(), so there is only one of each. Introduces about half of the actual cache logic, including a hashmap of cache file objects for bookkeeping and basic operations on it. 
> 
> 30039: Enables fetcher cache actions in the mesos fetcher program.
> 
> 30006: Enables concurrent downloading into the fetcher cache. Reuse of download results in the cache when multiple fetcher runs occur concurrently. 
> 
> 30614: This is to ensure that all this refactoring of fetcher code has not broken HDFS fetching. Adds a test that exercises the C++ code paths in Mesos and mesos-fetcher related to fetching from HDFS. Uses a mock HDFS client written in bash that acts just like a real "hadoop" command if used in the right limited way.
> 
> 30124: Inserted fetcher cache zap upon slave startup, recovery and shutdown. This implements recovery in an acceptable, yet most simple way.
> 
> 30173: Created fetcher cache tests. Adds a new test source file containing a test fixture and tests to find out if the fetcher cache works with a variety of settings.
> 
> 30616: Adds hdfs::du() which calls "hadoop fs -du -h" and returns a string that contains the file size for the URI passed as argument. This is needed to determine the size of a file on HDFS before downloading it to the fetcher cache (to ensure there is enough space).
> 
> 30621: Refactored URI type separation in mesos-fetcher. Moved the URI type separation code (distinguishes http, hdfs, local copying, etc.) from mesos-fetcher to the fetcher process/actor, since it is going to be reused by download size queries when we introduce fetcher cache management. Also factored out URI validation, which will be used the same way by mesos-fetcher and the fetcher process/actor.
> 
> 30626: Fetcher cache eviction. This happens when the cache does not have enough space to accomodate upcoming downloads to the cache. Necessary provisions included here:
> - mesos-fetcher does not run until evictions have been successful
> - Cache space is reserved while (async) waiting for eviction to succeed. If it fails, the reservation gets undone.
> - Reservations can be partly from available space, partly from evictions. All math included :-)
> - To find out how much space is needed, downloading has a prelude in which we query the download size from the URI. This works for all URI types that mesos-fetcher currently supports, including http and hdfs.
> - Size-determination requests are now synchronized, too. Only one per URI in play happens.
> - There is cleanup code for all kinds of error situations. At the very end of the fetch attempt, each list is processed for undoing things like space reservations and eviction disabling.
> - Eviction gets disabled for URIs that are currently in use, i.e. the related cache files are. We use reference counting for this, since there may be concurrent fetch attempts using the same cache files.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.

> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/launcher/fetcher.cpp, line 388
> > <https://reviews.apache.org/r/30774/diff/1/?file=857926#file857926line388>
> >
> >     Kill '} else {'.

That would be wrong. Then the CHECK would fire every time.


> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, line 75
> > <https://reviews.apache.org/r/30774/diff/1/?file=857931#file857931line75>
> >
> >     This error message is a bit little wierd since it's not that it can't be removed, it's that we can determine it's 'realpath'. Also, why not use that for the actual os::rmdir below?

This was accidental merge mess when I combined and rebased patches. Sorry! Cleaned up now.


> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/slave/containerizer/fetcher.cpp, line 323
> > <https://reviews.apache.org/r/30774/diff/1/?file=857931#file857931line323>
> >
> >     Were you explicitly trying to do this asynchronously? Since it could be an expensive operation (filesystem) I'm totally for it but just wanted to confirm.

Yes, any file operation needs to be async! I have sometimes encountered bad blocks on hard disks and experienced that when that happens any file operation that touches one may stall for minutes. Besides, we could be using a network-attached/mounted file system with arbitrarily long timeouts. Etc.


> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 220
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line220>
> >
> >     This should occur in the HttpServer but then you should have an HttpServerProcess. When does process terminate? How are you sure that libprocess is not calling something like 'visit' after the memory for this class has been cleaned up!!!

I added this comment to the class to clarify what the intent is here:
// A minimal HTTP server, not intended as an actor, just reusing what
// is already implemented somewhere to serve some HTTP requests for
// file downloads, plus counting how many requests are made.

There is an explicit terminate() call before Shutdown() in every test that uses this. I will add this to the fixture when creating one for this kind of test.

BTW, this is similar to this stretch of code in process_tests.cpp:

class FileServer : public Process<FileServer>
{
public:
  explicit FileServer(const string& _path)
    : path(_path) {}

  virtual void initialize()
  {
    provide("", path);
  }

  const string path;
};
...
  FileServer server(path);
  PID<FileServer> pid = spawn(server);

...
  terminate(server);
  wait(server);
---


Should I add wait() after terminate() in my tests?


> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 242
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line242>
> >
> >     I don't like the mixture of actors and threads here. Even if this is only in the tests, it sets a bad precedent that we're willing compromise on the integrity of the assumptions you have when executing within an actor.
> >     
> >     There is also a lot of expectations shared between the implementation here and the tests themselves. For example, 'pause' is meant to be used in order to enable queuing up multiple pending cache entry downloads.
> >     
> >     The abstraction I think you want here is something that lets you control what happens whenever we get a 'visit' callback, and to do that we really need to mock 'visit'. If you had a mocked 'visit' then you can explicitly control "how", and most importantly, "when" each 'visit' call occurs.

The HttpServer here is not an "actor". I am just reusing some code to produce an HttpServer as quickly as possible.

I think the problem I did miss is to establish that tasks have in fact queued up. Rather than just observing this when debugging. I may still need pausing the HTTP request handling then. Or at least some HTTP server to test the net::-related code paths in mesos-fetcher.

I don't see how a mocked method would have more control over when the fetcher does anything than an overridden method.


> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, line 438
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line438>
> >
> >     There is a ton of state, expectations, control flow requirements, etc, throughout these tests. This makes them very hard to follow, and thus hard to understand if you're really testing the things you want to test (i.e., concurrent requests, etc). While it's probably more code, I'd rather have seen each test case testing exactly what you wanted to test rather than a bunch of functions pulled out to cover "commonality". It makes it very difficult for me to determine if certain test situations are covered!
> >     
> >     For example, in this function you await tasks if in "concurrent" mode. Yet, I'm pretty sure tasks aren't going to run in parallel unless there are enough resources to do so, and tracking down the size of the tasks and how many resources were launched with the slave is so non-local that it requires looking all over this source file and after doing so I'm not convinced that anything can actually run in parallel since the default slave has 2 CPUs and each task takes 1 CPU (so maybe 2 tasks run in parallel but not all of them?).
> >     
> >     Why not be more explicit in the tests, even if it's more code, so that we're convinced we're testing the things we want to test? My guess is that there will still be things we can pull out and abstract after having done that.


> On Feb. 8, 2015, 2:06 p.m., Benjamin Hindman wrote:
> > src/tests/fetcher_cache_tests.cpp, lines 132-135
> > <https://reviews.apache.org/r/30774/diff/1/?file=857937#file857937line132>
> >
> >     There is only "checkpointing" mode now, so don't bother trying to test both cases.

I tried removing this distinction and tests failed, both the recovery test and the others that don't test it. Keeping the distinction for now.


- Bernd


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71575
-----------------------------------------------------------


On Feb. 8, 2015, 11:34 a.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 8, 2015, 11:34 a.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71575
-----------------------------------------------------------



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117341>

    Separate with newline please.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117342>

    '{' on previous line please.



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117343>

    Kill '} else {'.



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment117344>

    Alphabetical ordering please (move 'stout' down below 'process').



src/slave/containerizer/fetcher.hpp
<https://reviews.apache.org/r/30774/#comment117345>

    If we change the name here we should change the name in the Fetch class too please



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117347>

    This error message is a bit little wierd since it's not that it can't be removed, it's that we can determine it's 'realpath'. Also, why not use that for the actual os::rmdir below?



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117346>

    Fix indentation.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117349>

    auto downloads = make_shared<hashmap<string, Bytes>>();
    
    Also, I think using 'auto' here makes sense because the type information is redundant. Let's send an email to the mailing list with this as a proposal to use 'auto' in these contexts because the type information is redundant, where as the type information when we get something returned from a function is invaluable!



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117348>

    Were you explicitly trying to do this asynchronously? Since it could be an expensive operation (filesystem) I'm totally for it but just wanted to confirm.



src/slave/containerizer/fetcher.cpp
<https://reviews.apache.org/r/30774/#comment117350>

    Let's make HDFS::du() do more work for future people using it by moving this code into HDFS::du and having that return a Try<Bytes>.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117361>

    Is this supported by gcc 4.4? That's still the required compiler until after 0.22 is cut. How about we just make these be globals for now. same for the ones below too.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117362>

    CHECK_SOME(os::mkdir(...));



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117363>

    We just name the enum directly rather than with the typedef:
    
    enum ExecutionMode { ... };



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117370>

    There is only "checkpointing" mode now, so don't bother trying to test both cases.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117371>

    Please structure the test using the 'SetUp' 'TearDown' method overloads rather than handrolling this stuff yourself; it's far easier to follow when using 'SetUp' and 'TearDown' and they let you embed ASSERT_* and EXPECT_* cleanly.
    
    Note that when you need a slightly different setup/teardown flow then we either parameterize the test case or create more subclasses (e.g., subclasses of FetcherCacheTest such as ArchiveAssetFetcherCacheTest and ExecutableAssetFetcherCacheTest).



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117364>

    '{' on newline please.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117372>

    This should occur in the HttpServer but then you should have an HttpServerProcess. When does process terminate? How are you sure that libprocess is not calling something like 'visit' after the memory for this class has been cleaned up!!!



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117365>

    s/node/address/



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117373>

    I don't like the mixture of actors and threads here. Even if this is only in the tests, it sets a bad precedent that we're willing compromise on the integrity of the assumptions you have when executing within an actor.
    
    There is also a lot of expectations shared between the implementation here and the tests themselves. For example, 'pause' is meant to be used in order to enable queuing up multiple pending cache entry downloads.
    
    The abstraction I think you want here is something that lets you control what happens whenever we get a 'visit' callback, and to do that we really need to mock 'visit'. If you had a mocked 'visit' then you can explicitly control "how", and most importantly, "when" each 'visit' call occurs.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117379>

    We've got a concurrent queue in libprocess called 'Queue'. See scheduler_tests.cpp for how it's used. I'd love to see the 'Enqueue' gmock action be made more generic so that you can use it here too (I'm confident it can, and can help if need be, and would rather use that then introduce another concurrent queuing like data structure, even if it's only for the tests).



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117374>

    Why aren't we being more explicit about what we're checking for here?



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117377>

    There is a ton of state, expectations, control flow requirements, etc, throughout these tests. This makes them very hard to follow, and thus hard to understand if you're really testing the things you want to test (i.e., concurrent requests, etc). While it's probably more code, I'd rather have seen each test case testing exactly what you wanted to test rather than a bunch of functions pulled out to cover "commonality". It makes it very difficult for me to determine if certain test situations are covered!
    
    For example, in this function you await tasks if in "concurrent" mode. Yet, I'm pretty sure tasks aren't going to run in parallel unless there are enough resources to do so, and tracking down the size of the tasks and how many resources were launched with the slave is so non-local that it requires looking all over this source file and after doing so I'm not convinced that anything can actually run in parallel since the default slave has 2 CPUs and each task takes 1 CPU (so maybe 2 tasks run in parallel but not all of them?).
    
    Why not be more explicit in the tests, even if it's more code, so that we're convinced we're testing the things we want to test? My guess is that there will still be things we can pull out and abstract after having done that.



src/tests/fetcher_cache_tests.cpp
<https://reviews.apache.org/r/30774/#comment117378>

    How do you know that this index exists? Shouldn't we be doing a check before we loop through to make sure?


- Benjamin Hindman


On Feb. 8, 2015, 7:34 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 8, 2015, 7:34 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/#review71584
-----------------------------------------------------------


Bad patch!

Reviews applied: [30609, 30606, 30774]

Failed command: make -j3 distcheck

Error:
 make  dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot'
if test -d "mesos-0.22.0"; then find "mesos-0.22.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "mesos-0.22.0" || { sleep 5 && rm -rf "mesos-0.22.0"; }; else :; fi
test -d "mesos-0.22.0" || mkdir "mesos-0.22.0"
 (cd 3rdparty && make  top_distdir=../mesos-0.22.0 distdir=../mesos-0.22.0/3rdparty \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty'
 (cd libprocess && make  top_distdir=../../mesos-0.22.0 distdir=../../mesos-0.22.0/3rdparty/libprocess \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[3]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess'
:
test -d "../../mesos-0.22.0/3rdparty/libprocess" || mkdir "../../mesos-0.22.0/3rdparty/libprocess"
 (cd 3rdparty && make  top_distdir=../../../mesos-0.22.0 distdir=../../../mesos-0.22.0/3rdparty/libprocess/3rdparty \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty'
 (cd stout && make  top_distdir=../../../../mesos-0.22.0 distdir=../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout'
:
test -d "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout" || mkdir "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout"
 (cd include && make  top_distdir=../../../../../mesos-0.22.0 distdir=../../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout/include \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[6]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout/include'
make[6]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout/include'
test -n ":" \
	|| find "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout" -type d ! -perm -755 \
		-exec chmod u+rwx,go+rx {} \; -o \
	  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -444 -exec /bin/bash /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout/install-sh -c -m a+r {} {} \; \
	|| chmod -R a+r "../../../../mesos-0.22.0/3rdparty/libprocess/3rdparty/stout"
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty/stout'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/3rdparty'
 (cd include && make  top_distdir=../../../mesos-0.22.0 distdir=../../../mesos-0.22.0/3rdparty/libprocess/include \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/include'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/include'
test -n ":" \
	|| find "../../mesos-0.22.0/3rdparty/libprocess" -type d ! -perm -755 \
		-exec chmod u+rwx,go+rx {} \; -o \
	  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -444 -exec /bin/bash /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess/install-sh -c -m a+r {} {} \; \
	|| chmod -R a+r "../../mesos-0.22.0/3rdparty/libprocess"
make[3]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty/libprocess'
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/3rdparty'
 (cd src && make  top_distdir=../mesos-0.22.0 distdir=../mesos-0.22.0/src \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/src'
test ".." = ".." ||			\
		(/bin/mkdir -p python/src/mesos && cp -pf ./python/src/mesos/__init__.py python/src/mesos/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos && cp -pf ./python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos/interface && cp -pf ./python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos && cp -pf ./python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/module.cpp python/native/src/mesos/native/module.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/module.hpp python/native/src/mesos/native/module.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_executor.cpp python/native/src/mesos/native/proxy_executor.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_executor.hpp python/native/src/mesos/native/proxy_executor.hpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_scheduler.cpp python/native/src/mesos/native/proxy_scheduler.cpp)
test ".." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ./python/native/src/mesos/native/proxy_scheduler.hpp python/native/src/mesos/native/proxy_scheduler.hpp)
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/src'
 (cd ec2 && make  top_distdir=../mesos-0.22.0 distdir=../mesos-0.22.0/ec2 \
     am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/ec2'
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/ec2'
test -n "" \
	|| find "mesos-0.22.0" -type d ! -perm -755 \
		-exec chmod u+rwx,go+rx {} \; -o \
	  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
	  ! -type d ! -perm -444 -exec /bin/bash /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/install-sh -c -m a+r {} {} \; \
	|| chmod -R a+r "mesos-0.22.0"
tardir=mesos-0.22.0 && ${TAR-tar} chof - "$tardir" | GZIP=--best gzip -c >mesos-0.22.0.tar.gz
make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot'
if test -d "mesos-0.22.0"; then find "mesos-0.22.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "mesos-0.22.0" || { sleep 5 && rm -rf "mesos-0.22.0"; }; else :; fi
case 'mesos-0.22.0.tar.gz' in \
	*.tar.gz*) \
	  GZIP=--best gzip -dc mesos-0.22.0.tar.gz | ${TAR-tar} xf - ;;\
	*.tar.bz2*) \
	  bzip2 -dc mesos-0.22.0.tar.bz2 | ${TAR-tar} xf - ;;\
	*.tar.lz*) \
	  lzip -dc mesos-0.22.0.tar.lz | ${TAR-tar} xf - ;;\
	*.tar.xz*) \
	  xz -dc mesos-0.22.0.tar.xz | ${TAR-tar} xf - ;;\
	*.tar.Z*) \
	  uncompress -c mesos-0.22.0.tar.Z | ${TAR-tar} xf - ;;\
	*.shar.gz*) \
	  GZIP=--best gzip -dc mesos-0.22.0.shar.gz | unshar ;;\
	*.zip*) \
	  unzip mesos-0.22.0.zip ;;\
	esac
chmod -R a-w mesos-0.22.0
chmod u+w mesos-0.22.0
mkdir mesos-0.22.0/_build mesos-0.22.0/_inst
chmod a-w mesos-0.22.0
test -d mesos-0.22.0/_build || exit 0; \
	dc_install_base=`CDPATH="${ZSH_VERSION+.}:" && cd mesos-0.22.0/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \
	  && dc_destdir="${TMPDIR-/tmp}/am-dc-$$/" \
	  && am__cwd=`pwd` \
	  && CDPATH="${ZSH_VERSION+.}:" && cd mesos-0.22.0/_build \
	  && ../configure \
	    --disable-python-dependency-install \
	     \
	    --srcdir=.. --prefix="$dc_install_base" \
	  && make  \
	  && make  dvi \
	  && make  check \
	  && make  install \
	  && make  installcheck \
	  && make  uninstall \
	  && make  distuninstallcheck_dir="$dc_install_base" \
	        distuninstallcheck \
	  && chmod -R a-w "$dc_install_base" \
	  && ({ \
	       (cd ../.. && umask 077 && mkdir "$dc_destdir") \
	       && make  DESTDIR="$dc_destdir" install \
	       && make  DESTDIR="$dc_destdir" uninstall \
	       && make  DESTDIR="$dc_destdir" \
	            distuninstallcheck_dir="$dc_destdir" distuninstallcheck; \
	      } || { rm -rf "$dc_destdir"; exit 1; }) \
	  && rm -rf "$dc_destdir" \
	  && make  dist \
	  && rm -rf mesos-0.22.0.tar.gz \
	  && make  distcleancheck \
	  && cd "$am__cwd" \
	  || exit 1
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
configure: creating ./config.lt
config.lt: creating libtool
configure: Setting up build environment for x86_64 linux-gnu
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking if compiling with clang... no
configure: GCC version: 4.8
configure: Setting up CXXFLAGS for g++ version >= 4.8
checking whether g++ supports C++11 features by default... no
checking whether g++ supports C++11 features with -std=c++11... yes
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for backtrace in -lunwind... no
checking for gzread in -lz... yes
checking for curl_global_init in -lcurl... yes
checking apr_pools.h usability... yes
checking apr_pools.h presence... yes
checking for apr_pools.h... yes
checking for apr_initialize in -lapr-1... yes
checking svn_version.h usability... yes
checking svn_version.h presence... yes
checking for svn_version.h... yes
checking for svn_stringbuf_create_ensure in -lsvn_subr-1... yes
checking svn_delta.h usability... yes
checking svn_delta.h presence... yes
checking for svn_delta.h... yes
checking for svn_txdelta in -lsvn_delta-1... yes
checking for sasl_done in -lsasl2... yes
checking whether or not we can build with JNI... yes
checking for mvn... /home/jenkins/tools/maven/latest/bin/mvn
checking for a Python interpreter with version >= 2.6... python
checking for python... /usr/bin/python
checking for python version... 2.7
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages
checking for python2.7... (cached) /usr/bin/python
checking for a version of Python >= '2.1.0'... yes
checking for a version of Python >= '2.6'... yes
checking for the distutils Python package... yes
checking for Python include path... -I/usr/include/python2.7
checking for Python library path... -L/usr/lib -lpython2.7
checking for Python site-packages path... /usr/lib/python2.7/dist-packages
checking python extra libraries... -lpthread -ldl  -lutil
checking python extra linking flags... -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
checking consistency of all components of python development environment... yes
checking whether we can build usable Python eggs... cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled by default]
yes
checking for an old installation of the Mesos egg (before 0.20.0)... no
checking python2 module: boto... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating mesos.pc
config.status: creating ec2/Makefile
config.status: creating src/Makefile
config.status: creating 3rdparty/Makefile
config.status: creating bin/mesos.sh
config.status: creating bin/mesos-local.sh
config.status: creating bin/mesos-master.sh
config.status: creating bin/mesos-slave.sh
config.status: creating bin/mesos-tests.sh
config.status: creating bin/mesos-local-flags.sh
config.status: creating bin/mesos-master-flags.sh
config.status: creating bin/mesos-slave-flags.sh
config.status: creating bin/mesos-tests-flags.sh
config.status: creating bin/gdb-mesos-local.sh
config.status: creating bin/gdb-mesos-master.sh
config.status: creating bin/gdb-mesos-slave.sh
config.status: creating bin/gdb-mesos-tests.sh
config.status: creating bin/lldb-mesos-local.sh
config.status: creating bin/lldb-mesos-master.sh
config.status: creating bin/lldb-mesos-slave.sh
config.status: creating bin/lldb-mesos-tests.sh
config.status: creating bin/valgrind-mesos-local.sh
config.status: creating bin/valgrind-mesos-slave.sh
config.status: creating bin/valgrind-mesos-master.sh
config.status: creating bin/valgrind-mesos-tests.sh
config.status: creating src/deploy/mesos-daemon.sh
config.status: creating src/deploy/mesos-start-cluster.sh
config.status: creating src/deploy/mesos-start-masters.sh
config.status: creating src/deploy/mesos-start-slaves.sh
config.status: creating src/deploy/mesos-stop-cluster.sh
config.status: creating src/deploy/mesos-stop-masters.sh
config.status: creating src/deploy/mesos-stop-slaves.sh
config.status: creating include/mesos/version.hpp
config.status: creating src/java/generated/org/apache/mesos/MesosNativeLibrary.java
config.status: creating mpi/mpiexec-mesos
config.status: creating src/examples/java/test-exception-framework
config.status: creating src/examples/java/test-executor
config.status: creating src/examples/java/test-framework
config.status: creating src/examples/java/test-multiple-executors-framework
config.status: creating src/examples/java/test-log
config.status: creating src/java/mesos.pom
config.status: creating src/examples/python/test-executor
config.status: creating src/examples/python/test-framework
config.status: creating src/examples/python/test-containerizer
config.status: creating src/python/setup.py
config.status: creating src/python/interface/setup.py
config.status: creating src/python/native/ext_modules.py
config.status: creating src/python/native/setup.py
config.status: executing depfiles commands
config.status: executing libtool commands
=== configuring in 3rdparty/libprocess (/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess)
configure: running /bin/bash ../../../3rdparty/libprocess/configure --disable-option-checking '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst'  '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' --cache-file=/dev/null --srcdir=../../../3rdparty/libprocess
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
configure: creating ./config.lt
config.lt: creating libtool
checking apr_pools.h usability... yes
checking apr_pools.h presence... yes
checking for apr_pools.h... yes
checking for apr_initialize in -lapr-1... yes
checking svn_version.h usability... yes
checking svn_version.h presence... yes
checking for svn_version.h... yes
checking for svn_stringbuf_create_ensure in -lsvn_subr-1... yes
checking svn_delta.h usability... yes
checking svn_delta.h presence... yes
checking for svn_delta.h... yes
checking for svn_txdelta in -lsvn_delta-1... yes
checking for curl_global_init in -lcurl... yes
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking if compiling with clang... no
configure: GCC version: 4.8
configure: Setting up CXXFLAGS for g++-4.8
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for deflate, gzread, gzwrite, inflate in -lz... yes
checking whether g++ supports C++11 features by default... no
checking whether g++ supports C++11 features with -std=c++11... yes
checking for dlopen in -ldl... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating include/Makefile
config.status: creating 3rdparty/Makefile
config.status: creating 3rdparty/gmock_sources.cc
config.status: executing depfiles commands
config.status: executing libtool commands
=== configuring in 3rdparty/stout (/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout)
configure: running /bin/bash ../../../../../3rdparty/libprocess/3rdparty/stout/configure --disable-option-checking '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst'  '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' --cache-file=/dev/null --srcdir=../../../../../3rdparty/libprocess/3rdparty/stout
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating include/Makefile
make[1]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
Making all in .
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
make[2]: Nothing to be done for `all-am'.
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
Making all in 3rdparty
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
make  all-recursive
make[3]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
Making all in libprocess
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
Making all in 3rdparty
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/ry-http-parser-1c3624a.tar.gz | tar xf -
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/gmock-1.6.0.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/ry-http-parser-1c3624a.patch || patch -d ry-http-parser-1c3624a -p1 <../../../../3rdparty/libprocess/3rdparty/ry-http-parser-1c3624a.patch
touch ry-http-parser-1c3624a-stamp
test ! -e ../../../../3rdparty/libprocess/3rdparty/gmock-1.6.0.patch || patch -d gmock-1.6.0 -p1 <../../../../3rdparty/libprocess/3rdparty/gmock-1.6.0.patch
touch gmock-1.6.0-stamp
cd .. && /bin/bash ./config.status 3rdparty/gmock_sources.cc
config.status: creating 3rdparty/gmock_sources.cc
make  all-recursive
make[6]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
Making all in stout
make[7]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
Making all in .
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
make[8]: Nothing to be done for `all-am'.
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
Making all in include
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout/include'
make[8]: Nothing to be done for `all'.
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout/include'
make[7]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/stout'
make[7]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../../3rdparty/libprocess/3rdparty  -Iry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -MT libry_http_parser_la-http_parser.lo -MD -MP -MF .deps/libry_http_parser_la-http_parser.Tpo -c -o libry_http_parser_la-htt
 p_parser.lo `test -f 'ry-http-parser-1c3624a/http_parser.c' || echo '../../../../3rdparty/libprocess/3rdparty/'`ry-http-parser-1c3624a/http_parser.c
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/protobuf-2.5.0.tar.gz | tar xf -
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/libev-4.15.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/libev-4.15.patch || patch -d libev-4.15 -p1 <../../../../3rdparty/libprocess/3rdparty/libev-4.15.patch
patching file ev.h
touch libev-4.15-stamp
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/boost-1.53.0.tar.gz | tar xf -
libtool: compile:  gcc -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../../3rdparty/libprocess/3rdparty -Iry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -g1 -O0 -Wno-unused-local-typedefs -MT libry_http_parser_la-http_parser.lo -MD -MP -MF .deps/libry_http_parser_la-http_parser.Tpo -c ry-http-parser-1c3624a/http_parser.c  -fPIC -DPIC -o libry
 _http_parser_la-http_parser.o
test ! -e ../../../../3rdparty/libprocess/3rdparty/protobuf-2.5.0.patch || patch -d protobuf-2.5.0 -p1 <../../../../3rdparty/libprocess/3rdparty/protobuf-2.5.0.patch
touch protobuf-2.5.0-stamp
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/picojson-4f93734.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/picojson-4f93734.patch || patch -d picojson-4f93734 -p1 <../../../../3rdparty/libprocess/3rdparty/picojson-4f93734.patch
touch picojson-4f93734-stamp
gzip -d -c ../../../../3rdparty/libprocess/3rdparty/glog-0.3.3.tar.gz | tar xf -
test ! -e ../../../../3rdparty/libprocess/3rdparty/boost-1.53.0.patch || patch -d boost-1.53.0 -p1 <../../../../3rdparty/libprocess/3rdparty/boost-1.53.0.patch
touch boost-1.53.0-stamp
cd protobuf-2.5.0 && ./configure '--disable-option-checking' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' '--cache-file=/dev/null' '--srcdir=../../../3rdparty/libprocess' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
          make 
test ! -e ../../../../3rdparty/libprocess/3rdparty/glog-0.3.3.patch || patch -d glog-0.3.3 -p1 <../../../../3rdparty/libprocess/3rdparty/glog-0.3.3.patch
patching file src/symbolize.cc
patching file src/glog/stl_logging.h.in
patching file src/stl_logging_unittest.cc
patching file src/demangle.cc
touch glog-0.3.3-stamp
cd libev-4.15 && ./configure '--disable-option-checking' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' '--cache-file=/dev/null' '--srcdir=../../../3rdparty/libprocess' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
          make 
mv -f .deps/libry_http_parser_la-http_parser.Tpo .deps/libry_http_parser_la-http_parser.Plo
cd glog-0.3.3 && ./configure GTEST_CONFIG=no '--disable-option-checking' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' '--enable-shared=no' '--with-pic' '--cache-file=/dev/null' '--srcdir=../../../3rdparty/libprocess' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
          make 
checking whether to enable maintainer-specific portions of Makefiles... yes
checking build system type... checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... checking whether to enable maintainer-specific portions of Makefiles... no
checking for gcc... gcc
yes
yes
checking for a thread-safe mkdir -p... checking for gcc... gcc
/bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether the C compiler works... checking for gcc... gcc
checking whether the C compiler works... checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... 
checking whether we are cross compiling... 
checking whether we are cross compiling... no
checking for suffix of object files... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... yes
checking for gcc option to accept ISO C89... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
gcc3
checking how to run the C preprocessor... checking build system type... gcc3
checking for g++... g++
gcc -E
checking whether we are using the GNU C++ compiler... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... checking for g++... g++
/bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... yes
checking whether g++ accepts -g... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking whether we are using the GNU C++ compiler... checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... yes
checking dependency style of g++... yes
checking whether g++ accepts -g... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... yes
checking dependency style of g++... gcc3
checking how to run the C++ preprocessor... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... g++ -E
gcc3
checking build system type... checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... ok
/bin/grep -F
checking for ld used by gcc... checking for sysroot... no
/usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... gcc -E
yes
checking for sys/types.h... checking for ANSI C header files... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... yes
checking for sys/stat.h... yes
checking for stdlib.h... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
yes
yes
checking for ANSI C header files... checking for sys/types.h... checking for string.h... no
yes
yes
checking for sys/types.h... checking for sys/stat.h... checking for memory.h... yes
checking for sys/stat.h... yes
yes
checking for stdlib.h... checking for strings.h... yes
checking for stdlib.h... yes
yes
checking for string.h... checking for inttypes.h... yes
checking for string.h... yes
yes
checking for memory.h... checking for stdint.h... yes
checking for memory.h... yes
yes
checking for unistd.h... checking for strings.h... yes
checking for strings.h... yes
yes
checking minix/config.h usability... yes
checking for inttypes.h... checking for inttypes.h... yes
yes
checking for stdint.h... checking for stdint.h... no
checking minix/config.h presence... yes
yes
checking for unistd.h... checking for unistd.h... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
yes
checking C++ compiler flags...... use user-supplied: -g1 -O0 -Wno-unused-local-typedefs -std=c++11
checking for dlfcn.h... yes
checking whether __SUNPRO_CC is declared... checking for dlfcn.h... yes
checking for objdir... no
checking how to print strings... .libs
yes
checking for objdir... printf
checking for a sed that does not truncate output... .libs
/bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... checking if gcc supports -fno-rtti -fno-exceptions... checking if gcc supports -fno-rtti -fno-exceptions... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc static flag -static works... ok
checking for sysroot... no
yes
checking if gcc supports -c -o file.o... checking for mt... mt
checking if mt is a manifest tool... yes
checking if gcc supports -c -o file.o... no
checking for dlfcn.h... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... yes
checking for objdir... .libs
GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking sys/inotify.h usability... g++ -E
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking sys/inotify.h presence... yes
checking for sys/inotify.h... yes
checking sys/epoll.h usability... yes
checking if gcc static flag -static works... checking for ld used by g++... yes
checking sys/epoll.h presence... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
yes
checking for sys/epoll.h... yes
yes
checking if gcc supports -c -o file.o... checking sys/event.h usability... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if g++ static flag -static works... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
no
checking sys/event.h presence... no
checking for sys/event.h... no
checking port.h usability... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) no
checking for stdint.h... (cached) yes
checking for sys/types.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for unistd.h... (cached) yes
checking for ld used by g++... checking syscall.h usability... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
yes
checking syscall.h presence... yes
checking for syscall.h... yes
checking sys/syscall.h usability... checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... no
checking port.h presence... yes
checking sys/syscall.h presence... yes
checking if g++ static flag -static works... no
checking for port.h... no
checking poll.h usability... yes
checking for sys/syscall.h... yes
checking execinfo.h usability... yes
checking poll.h presence... yes
checking execinfo.h presence... yes
checking for poll.h... yes
yes
checking for execinfo.h... yes
checking libunwind.h usability... yes
checking if g++ supports -c -o file.o... checking sys/select.h usability... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) yes
checking fcntl.h usability... yes
checking sys/select.h presence... yes
checking for sys/select.h... yes
checking sys/eventfd.h usability... yes
checking fcntl.h presence... no
checking libunwind.h presence... yes
checking for fcntl.h... yes
no
checking for libunwind.h... no
checking ucontext.h usability... checking for inttypes.h... (cached) yes
yes
checking sys/eventfd.h presence... checking limits.h usability... yes
checking for sys/eventfd.h... yes
checking sys/signalfd.h usability... yes
checking ucontext.h presence... yes
checking limits.h presence... yes
checking for ucontext.h... yes
checking sys/utsname.h usability... yes
checking for limits.h... yes
checking for stdlib.h... (cached) yes
yes
checking sys/signalfd.h presence... checking for unistd.h... (cached) yes
checking for working memcmp... yes
checking for sys/signalfd.h... yes
checking for inotify_init... yes
checking sys/utsname.h presence... yes
checking for sys/utsname.h... yes
checking pwd.h usability... yes
checking for epoll_ctl... yes
checking pwd.h presence... yes
checking for working strtod... yes
checking for pwd.h... yes
checking syslog.h usability... yes
checking for kqueue... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking sys/time.h usability... yes
checking for ftruncate... no
checking for port_create... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
yes
checking glob.h usability... checking for memset... no
checking for poll... yes
checking glob.h presence... yes
checking for glob.h... yes
checking unwind.h usability... yes
yes
checking for mkdir... checking for select... yes
checking unwind.h presence... yes
checking for unwind.h... yes
yes
checking size of void *... yes
checking for eventfd... checking for strchr... yes
checking for signalfd... yes
checking for strerror... 8
checking for uint16_t... yes
checking for clock_gettime... yes
checking for strtol... yes
checking for nanosleep... yes
checking for u_int16_t... yes
checking zlib version... yes
checking for library containing floor... ok (1.2.0.4 or later)
checking for library containing zlibVersion... yes
checking for __uint16... -lm
no
checking for sigaltstack... -lz
configure: creating ./config.status
checking for the pthreads library -lpthreads... yes
checking for dladdr... no
checking whether pthreads work without any flags... no
checking for fcntl... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... yes
checking for __attribute__... no
checking whether pthreads work with -pthread... yes
checking for __builtin_expect... yes
checking for __sync_val_compare_and_swap... yes
checking for joinable pthread attribute... yes
PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... checking for the pthreads library -lpthreads... yes
checking whether -pthread is sufficient with -shared... no
checking whether pthreads work without any flags... yes
checking whether what we have so far is sufficient with -nostdlib... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether -lpthread saves the day... no
checking whether pthreads work with -pthread... yes
checking the location of hash_map... config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
yes
config.status: executing libtool commands
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for pthread_self in -lpthread... make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
make  all-am
make[9]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
/bin/bash ./libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.     -g -O3 -MT ev.lo -MD -MP -MF .deps/ev.Tpo -c -o ev.lo ev.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O3 -MT ev.lo -MD -MP -MF .deps/ev.Tpo -c ev.c  -fPIC -DPIC -o ev.o
yes
checking for main in -lgflags... no
checking for gtest-config... no
checking support for pthread_rwlock_* functions... ev.c:1531:31: warning: 'ev_default_loop_ptr' initialized and declared 'extern' [enabled by default]
   EV_API_DECL struct ev_loop *ev_default_loop_ptr = 0; /* needs to be initialised to make it a definition despite extern */
                               ^
yes
checking whether the compiler implements namespaces... ev.c: In function 'evpipe_write':
ev.c:2160:17: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result]
           write (evpipe [1], &counter, sizeof (uint64_t));
                 ^
ev.c:2172:17: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result]
           write (evpipe [1], &(evpipe [1]), 1);
                 ^
ev.c: In function 'pipecb':
ev.c:2193:16: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result]
           read (evpipe [1], &counter, sizeof (uint64_t));
                ^
ev.c:2207:16: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result]
           read (evpipe [0], &dummy, sizeof (dummy));
                ^
yes
checking what namespace STL code is in... <unordered_map>
configure: creating ./config.status
std
checking whether compiler supports using ::operator<<... 1
checking for ucontext.h... (cached) yes
checking sys/ucontext.h usability... yes
checking sys/ucontext.h presence... yes
checking for sys/ucontext.h... yes
checking how to access the program counter from a struct ucontext... uc_mcontext.gregs[REG_RIP]
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating protobuf.pc
configure: creating ./config.status
config.status: creating protobuf-lite.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
mv -f .deps/ev.Tpo .deps/ev.Plo
/bin/bash ./libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.     -g -O3 -MT event.lo -MD -MP -MF .deps/event.Tpo -c -o event.lo event.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O3 -MT event.lo -MD -MP -MF .deps/event.Tpo -c event.c  -fPIC -DPIC -o event.o
=== configuring in gtest (/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/gtest)
configure: running /bin/bash ./configure --disable-option-checking '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst'  '--disable-python-dependency-install' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' 'CXXFLAGS=-g1 -O0 -Wno-unused-local-typedefs -std=c++11' '--enable-shared=no' '--with-pic' --cache-file=/dev/null --srcdir=.
mv -f .deps/event.Tpo .deps/event.Plo
/bin/bash ./libtool --tag=CC   --mode=link gcc  -g -O3 -version-info 4:0:0  -o libev.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib ev.lo event.lo  -lm 
libtool: link: ar cru .libs/libev.a  ev.o event.o
libtool: link: ranlib .libs/libev.a
checking for a BSD-compatible install... libtool: link: ( cd ".libs" && rm -f "libev.la" && ln -s "../libev.la" "libev.la" )
make[9]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/libev-4.15'
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g1 -O0 -Wno-unused-local-typedefs   -o libry_http_parser.la  libry_http_parser_la-http_parser.lo  -ldl -lz -lcurl -lsvn_delta-1 -lsvn_subr-1 -lapr-1  -lrt
/usr/bin/install -c
checking whether build environment is sane... config.status: creating Makefile
config.status: creating src/glog/logging.h
config.status: creating src/glog/raw_logging.h
libtool: link: ar cru .libs/libry_http_parser.a  libry_http_parser_la-http_parser.o
libtool: link: ranlib .libs/libry_http_parser.a
libtool: link: ( cd ".libs" && rm -f "libry_http_parser.la" && ln -s "../libry_http_parser.la" "libry_http_parser.la" )
config.status: creating src/glog/vlog_is_on.h
config.status: creating src/glog/stl_logging.h
config.status: creating libglog.pc
config.status: creating src/config.h
config.status: executing depfiles commands
config.status: executing libtool commands
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/glog-0.3.3'
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-logging.lo -MD -MP -MF .deps/libglog_la-logging.Tpo -c -o libglog_la-logging.lo `test -f 'src/logging.cc' || echo './'`src/logging.cc
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-raw_logging.lo -MD -MP -MF .deps/libglog_la-raw_logging.Tpo -c -o libglog_la-raw_logging.lo `test -f 'src/raw_logging.cc' || echo './'`src/raw_logging.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-logging.lo -MD -MP -MF .deps/libglog_la-logging.Tpo -c src/logging.cc  -fPIC -DPIC -o libglog_la-logging.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-raw_logging.lo -MD -MP -MF .deps/libglog_la-raw_logging.Tpo -c src/raw_logging.cc  -fPIC -DPIC -o libglog_la-raw_logging.o
mv -f .deps/libglog_la-raw_logging.Tpo .deps/libglog_la-raw_logging.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-vlog_is_on.lo -MD -MP -MF .deps/libglog_la-vlog_is_on.Tpo -c -o libglog_la-vlog_is_on.lo `test -f 'src/vlog_is_on.cc' || echo './'`src/vlog_is_on.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-vlog_is_on.lo -MD -MP -MF .deps/libglog_la-vlog_is_on.Tpo -c src/vlog_is_on.cc  -fPIC -DPIC -o libglog_la-vlog_is_on.o
yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... mv -f .deps/libglog_la-vlog_is_on.Tpo .deps/libglog_la-vlog_is_on.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-utilities.lo -MD -MP -MF .deps/libglog_la-utilities.Tpo -c -o libglog_la-utilities.lo `test -f 'src/utilities.cc' || echo './'`src/utilities.cc
gcc3
checking for g++... g++
checking whether we are using the GNU C++ compiler... libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-utilities.lo -MD -MP -MF .deps/libglog_la-utilities.Tpo -c src/utilities.cc  -fPIC -DPIC -o libglog_la-utilities.o
yes
checking whether g++ accepts -g... yes
checking dependency style of g++... mv -f .deps/libglog_la-logging.Tpo .deps/libglog_la-logging.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-demangle.lo -MD -MP -MF .deps/libglog_la-demangle.Tpo -c -o libglog_la-demangle.lo `test -f 'src/demangle.cc' || echo './'`src/demangle.cc
gcc3
checking build system type... libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-demangle.lo -MD -MP -MF .deps/libglog_la-demangle.Tpo -c src/demangle.cc  -fPIC -DPIC -o libglog_la-demangle.o
x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... mv -f .deps/libglog_la-demangle.Tpo .deps/libglog_la-demangle.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-symbolize.lo -MD -MP -MF .deps/libglog_la-symbolize.Tpo -c -o libglog_la-symbolize.lo `test -f 'src/symbolize.cc' || echo './'`src/symbolize.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-symbolize.lo -MD -MP -MF .deps/libglog_la-symbolize.Tpo -c src/symbolize.cc  -fPIC -DPIC -o libglog_la-symbolize.o
ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... mv -f .deps/libglog_la-utilities.Tpo .deps/libglog_la-utilities.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -I./src    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-signalhandler.lo -MD -MP -MF .deps/libglog_la-signalhandler.Tpo -c -o libglog_la-signalhandler.lo `test -f 'src/signalhandler.cc' || echo './'`src/signalhandler.cc
gcc -E
checking for ANSI C header files... libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -I./src -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libglog_la-signalhandler.lo -MD -MP -MF .deps/libglog_la-signalhandler.Tpo -c src/signalhandler.cc  -fPIC -DPIC -o libglog_la-signalhandler.o
yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... mv -f .deps/libglog_la-symbolize.Tpo .deps/libglog_la-symbolize.Plo
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_unittest-logging_unittest.o -MD -MP -MF .deps/logging_unittest-logging_unittest.Tpo -c -o logging_unittest-logging_unittest.o `test -f 'src/logging_unittest.cc' || echo './'`src/logging_unittest.cc
yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... mv -f .deps/libglog_la-signalhandler.Tpo .deps/libglog_la-signalhandler.Plo
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT demangle_unittest-demangle_unittest.o -MD -MP -MF .deps/demangle_unittest-demangle_unittest.Tpo -c -o demangle_unittest-demangle_unittest.o `test -f 'src/demangle_unittest.cc' || echo './'`src/demangle_unittest.cc
yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for python... /usr/bin/python
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... mv -f .deps/demangle_unittest-demangle_unittest.Tpo .deps/demangle_unittest-demangle_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stacktrace_unittest-stacktrace_unittest.o -MD -MP -MF .deps/stacktrace_unittest-stacktrace_unittest.Tpo -c -o stacktrace_unittest-stacktrace_unittest.o `test -f 'src/stacktrace_unittest.cc' || echo './'`src/stacktrace_unittest.cc
no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
configure: creating ./config.status
mv -f .deps/stacktrace_unittest-stacktrace_unittest.Tpo .deps/stacktrace_unittest-stacktrace_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT symbolize_unittest-symbolize_unittest.o -MD -MP -MF .deps/symbolize_unittest-symbolize_unittest.Tpo -c -o symbolize_unittest-symbolize_unittest.o `test -f 'src/symbolize_unittest.cc' || echo './'`src/symbolize_unittest.cc
mv -f .deps/logging_unittest-logging_unittest.Tpo .deps/logging_unittest-logging_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stl_logging_unittest-stl_logging_unittest.o -MD -MP -MF .deps/stl_logging_unittest-stl_logging_unittest.Tpo -c -o stl_logging_unittest-stl_logging_unittest.o `test -f 'src/stl_logging_unittest.cc' || echo './'`src/stl_logging_unittest.cc
In file included from /usr/include/c++/4.8/ext/hash_set:60:0,
                 from src/glog/stl_logging.h:54,
                 from src/stl_logging_unittest.cc:34:
/usr/include/c++/4.8/backward/backward_warning.h:32:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp]
 #warning \
  ^
In file included from src/utilities.h:73:0,
                 from src/googletest.h:38,
                 from src/stl_logging_unittest.cc:48:
src/base/mutex.h:137:0: warning: "_XOPEN_SOURCE" redefined [enabled by default]
 #   define _XOPEN_SOURCE 500  // may be needed to get the rwlock calls
 ^
In file included from /usr/include/x86_64-linux-gnu/c++/4.8/bits/os_defines.h:39:0,
                 from /usr/include/x86_64-linux-gnu/c++/4.8/bits/c++config.h:426,
                 from /usr/include/c++/4.8/bits/stl_algobase.h:59,
                 from /usr/include/c++/4.8/deque:60,
                 from src/glog/stl_logging.h:45,
                 from src/stl_logging_unittest.cc:34:
/usr/include/features.h:162:0: note: this is the location of the previous definition
 # define _XOPEN_SOURCE 700
 ^
config.status: creating Makefile
config.status: creating scripts/gtest-config
config.status: creating build-aux/config.h
config.status: executing depfiles commands
mv -f .deps/symbolize_unittest-symbolize_unittest.Tpo .deps/symbolize_unittest-symbolize_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT utilities_unittest-utilities_unittest.o -MD -MP -MF .deps/utilities_unittest-utilities_unittest.Tpo -c -o utilities_unittest-utilities_unittest.o `test -f 'src/utilities_unittest.cc' || echo './'`src/utilities_unittest.cc
config.status: executing libtool commands
make[8]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
make  all-recursive
make[9]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
Making all in .
make[10]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
make[10]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
Making all in src
make[10]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o `test -f 'google/protobuf/compiler/main.cc' || echo './'`google/protobuf/compiler/main.cc
mv -f .deps/utilities_unittest-utilities_unittest.Tpo .deps/utilities_unittest-utilities_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_striptest0-logging_striptest_main.o -MD -MP -MF .deps/logging_striptest0-logging_striptest_main.Tpo -c -o logging_striptest0-logging_striptest_main.o `test -f 'src/logging_striptest_main.cc' || echo './'`src/logging_striptest_main.cc
mv -f .deps/main.Tpo .deps/main.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_gcc.lo -MD -MP -MF .deps/atomicops_internals_x86_gcc.Tpo -c -o atomicops_internals_x86_gcc.lo `test -f 'google/protobuf/stubs/atomicops_internals_x86_gcc.cc' || echo './'`google/protobuf/stubs/atomicops_internals_x86_gcc.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_gcc.lo -MD -MP -MF .deps/atomicops_internals_x86_gcc.Tpo -c google/protobuf/stubs/atomicops_internals_x86_gcc.cc  -fPIC -DPIC -o atomicops_internals_x86_gcc.o
mv -f .deps/atomicops_internals_x86_gcc.Tpo .deps/atomicops_internals_x86_gcc.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_msvc.lo -MD -MP -MF .deps/atomicops_internals_x86_msvc.Tpo -c -o atomicops_internals_x86_msvc.lo `test -f 'google/protobuf/stubs/atomicops_internals_x86_msvc.cc' || echo './'`google/protobuf/stubs/atomicops_internals_x86_msvc.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT atomicops_internals_x86_msvc.lo -MD -MP -MF .deps/atomicops_internals_x86_msvc.Tpo -c google/protobuf/stubs/atomicops_internals_x86_msvc.cc  -fPIC -DPIC -o atomicops_internals_x86_msvc.o
src/logging_striptest_main.cc: In function 'int CheckNoReturn(bool)':
src/logging_striptest_main.cc:54:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
mv -f .deps/logging_striptest0-logging_striptest_main.Tpo .deps/logging_striptest0-logging_striptest_main.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_striptest2-logging_striptest2.o -MD -MP -MF .deps/logging_striptest2-logging_striptest2.Tpo -c -o logging_striptest2-logging_striptest2.o `test -f 'src/logging_striptest2.cc' || echo './'`src/logging_striptest2.cc
mv -f .deps/atomicops_internals_x86_msvc.Tpo .deps/atomicops_internals_x86_msvc.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common.lo -MD -MP -MF .deps/common.Tpo -c -o common.lo `test -f 'google/protobuf/stubs/common.cc' || echo './'`google/protobuf/stubs/common.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common.lo -MD -MP -MF .deps/common.Tpo -c google/protobuf/stubs/common.cc  -fPIC -DPIC -o common.o
mv -f .deps/stl_logging_unittest-stl_logging_unittest.Tpo .deps/stl_logging_unittest-stl_logging_unittest.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging_striptest10-logging_striptest10.o -MD -MP -MF .deps/logging_striptest10-logging_striptest10.Tpo -c -o logging_striptest10-logging_striptest10.o `test -f 'src/logging_striptest10.cc' || echo './'`src/logging_striptest10.cc
In file included from src/logging_striptest2.cc:35:0:
src/logging_striptest_main.cc: In function 'int CheckNoReturn(bool)':
src/logging_striptest_main.cc:54:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
mv -f .deps/logging_striptest2-logging_striptest2.Tpo .deps/logging_striptest2-logging_striptest2.Po
g++ -DHAVE_CONFIG_H -I. -I./src  -I./src  -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT signalhandler_unittest-signalhandler_unittest.o -MD -MP -MF .deps/signalhandler_unittest-signalhandler_unittest.Tpo -c -o signalhandler_unittest-signalhandler_unittest.o `test -f 'src/signalhandler_unittest.cc' || echo './'`src/signalhandler_unittest.cc
mv -f .deps/common.Tpo .deps/common.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT once.lo -MD -MP -MF .deps/once.Tpo -c -o once.lo `test -f 'google/protobuf/stubs/once.cc' || echo './'`google/protobuf/stubs/once.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT once.lo -MD -MP -MF .deps/once.Tpo -c google/protobuf/stubs/once.cc  -fPIC -DPIC -o once.o
In file included from src/logging_striptest10.cc:35:0:
src/logging_striptest_main.cc: In function 'int CheckNoReturn(bool)':
src/logging_striptest_main.cc:54:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
mv -f .deps/logging_striptest10-logging_striptest10.Tpo .deps/logging_striptest10-logging_striptest10.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stringprintf.lo -MD -MP -MF .deps/stringprintf.Tpo -c -o stringprintf.lo `test -f 'google/protobuf/stubs/stringprintf.cc' || echo './'`google/protobuf/stubs/stringprintf.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT stringprintf.lo -MD -MP -MF .deps/stringprintf.Tpo -c google/protobuf/stubs/stringprintf.cc  -fPIC -DPIC -o stringprintf.o
mv -f .deps/signalhandler_unittest-signalhandler_unittest.Tpo .deps/signalhandler_unittest-signalhandler_unittest.Po
/bin/bash ./libtool --tag=CXX   --mode=link g++   -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -DNDEBUG -g1 -O0 -Wno-unused-local-typedefs -std=c++11    -o libglog.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib  libglog_la-logging.lo libglog_la-raw_logging.lo libglog_la-vlog_is_on.lo libglog_la-utilities.lo libglog_la-demangle.lo libglog_la-symbolize.lo libglog_la-signalhandler.lo     -lpthread 
mv -f .deps/once.Tpo .deps/once.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set.lo -MD -MP -MF .deps/extension_set.Tpo -c -o extension_set.lo `test -f 'google/protobuf/extension_set.cc' || echo './'`google/protobuf/extension_set.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set.lo -MD -MP -MF .deps/extension_set.Tpo -c google/protobuf/extension_set.cc  -fPIC -DPIC -o extension_set.o
libtool: link: ar cru .libs/libglog.a  libglog_la-logging.o libglog_la-raw_logging.o libglog_la-vlog_is_on.o libglog_la-utilities.o libglog_la-demangle.o libglog_la-symbolize.o libglog_la-signalhandler.o
libtool: link: ranlib .libs/libglog.a
libtool: link: ( cd ".libs" && rm -f "libglog.la" && ln -s "../libglog.la" "libglog.la" )
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_unittest  logging_unittest-logging_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_unittest logging_unittest-logging_unittest.o  ./.libs/libglog.a -lpthread -pthread
mv -f .deps/stringprintf.Tpo .deps/stringprintf.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_util.lo -MD -MP -MF .deps/generated_message_util.Tpo -c -o generated_message_util.lo `test -f 'google/protobuf/generated_message_util.cc' || echo './'`google/protobuf/generated_message_util.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_util.lo -MD -MP -MF .deps/generated_message_util.Tpo -c google/protobuf/generated_message_util.cc  -fPIC -DPIC -o generated_message_util.o
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o demangle_unittest  demangle_unittest-demangle_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o demangle_unittest demangle_unittest-demangle_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o stacktrace_unittest  stacktrace_unittest-stacktrace_unittest.o  libglog.la    -lpthread 
mv -f .deps/generated_message_util.Tpo .deps/generated_message_util.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message_lite.lo -MD -MP -MF .deps/message_lite.Tpo -c -o message_lite.lo `test -f 'google/protobuf/message_lite.cc' || echo './'`google/protobuf/message_lite.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message_lite.lo -MD -MP -MF .deps/message_lite.Tpo -c google/protobuf/message_lite.cc  -fPIC -DPIC -o message_lite.o
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o stacktrace_unittest stacktrace_unittest-stacktrace_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o symbolize_unittest  symbolize_unittest-symbolize_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o symbolize_unittest symbolize_unittest-symbolize_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o stl_logging_unittest  stl_logging_unittest-stl_logging_unittest.o  libglog.la       -lpthread 
mv -f .deps/message_lite.Tpo .deps/message_lite.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT repeated_field.lo -MD -MP -MF .deps/repeated_field.Tpo -c -o repeated_field.lo `test -f 'google/protobuf/repeated_field.cc' || echo './'`google/protobuf/repeated_field.cc
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o stl_logging_unittest stl_logging_unittest-stl_logging_unittest.o  ./.libs/libglog.a -lpthread -pthread
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT repeated_field.lo -MD -MP -MF .deps/repeated_field.Tpo -c google/protobuf/repeated_field.cc  -fPIC -DPIC -o repeated_field.o
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o utilities_unittest  utilities_unittest-utilities_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o utilities_unittest utilities_unittest-utilities_unittest.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_striptest0  logging_striptest0-logging_striptest_main.o  libglog.la    -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_striptest0 logging_striptest0-logging_striptest_main.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_striptest2  logging_striptest2-logging_striptest2.o  libglog.la    -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_striptest2 logging_striptest2-logging_striptest2.o  ./.libs/libglog.a -lpthread -pthread
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o logging_striptest10  logging_striptest10-logging_striptest10.o  libglog.la    -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o logging_striptest10 logging_striptest10-logging_striptest10.o  ./.libs/libglog.a -lpthread -pthread
mv -f .deps/repeated_field.Tpo .deps/repeated_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format_lite.lo -MD -MP -MF .deps/wire_format_lite.Tpo -c -o wire_format_lite.lo `test -f 'google/protobuf/wire_format_lite.cc' || echo './'`google/protobuf/wire_format_lite.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format_lite.lo -MD -MP -MF .deps/wire_format_lite.Tpo -c google/protobuf/wire_format_lite.cc  -fPIC -DPIC -o wire_format_lite.o
/bin/bash ./libtool --tag=CXX   --mode=link g++ -pthread    -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -DNO_FRAME_POINTER  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o signalhandler_unittest  signalhandler_unittest-signalhandler_unittest.o  libglog.la       -lpthread 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -DNO_FRAME_POINTER -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o signalhandler_unittest signalhandler_unittest-signalhandler_unittest.o  ./.libs/libglog.a -lpthread -pthread
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/glog-0.3.3'
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT coded_stream.lo -MD -MP -MF .deps/coded_stream.Tpo -c -o coded_stream.lo `test -f 'google/protobuf/io/coded_stream.cc' || echo './'`google/protobuf/io/coded_stream.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT coded_stream.lo -MD -MP -MF .deps/coded_stream.Tpo -c google/protobuf/io/coded_stream.cc  -fPIC -DPIC -o coded_stream.o
mv -f .deps/extension_set.Tpo .deps/extension_set.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream.lo -MD -MP -MF .deps/zero_copy_stream.Tpo -c -o zero_copy_stream.lo `test -f 'google/protobuf/io/zero_copy_stream.cc' || echo './'`google/protobuf/io/zero_copy_stream.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream.lo -MD -MP -MF .deps/zero_copy_stream.Tpo -c google/protobuf/io/zero_copy_stream.cc  -fPIC -DPIC -o zero_copy_stream.o
mv -f .deps/zero_copy_stream.Tpo .deps/zero_copy_stream.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl_lite.lo -MD -MP -MF .deps/zero_copy_stream_impl_lite.Tpo -c -o zero_copy_stream_impl_lite.lo `test -f 'google/protobuf/io/zero_copy_stream_impl_lite.cc' || echo './'`google/protobuf/io/zero_copy_stream_impl_lite.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl_lite.lo -MD -MP -MF .deps/zero_copy_stream_impl_lite.Tpo -c google/protobuf/io/zero_copy_stream_impl_lite.cc  -fPIC -DPIC -o zero_copy_stream_impl_lite.o
mv -f .deps/wire_format_lite.Tpo .deps/wire_format_lite.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT strutil.lo -MD -MP -MF .deps/strutil.Tpo -c -o strutil.lo `test -f 'google/protobuf/stubs/strutil.cc' || echo './'`google/protobuf/stubs/strutil.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT strutil.lo -MD -MP -MF .deps/strutil.Tpo -c google/protobuf/stubs/strutil.cc  -fPIC -DPIC -o strutil.o
mv -f .deps/coded_stream.Tpo .deps/coded_stream.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT substitute.lo -MD -MP -MF .deps/substitute.Tpo -c -o substitute.lo `test -f 'google/protobuf/stubs/substitute.cc' || echo './'`google/protobuf/stubs/substitute.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT substitute.lo -MD -MP -MF .deps/substitute.Tpo -c google/protobuf/stubs/substitute.cc  -fPIC -DPIC -o substitute.o
mv -f .deps/zero_copy_stream_impl_lite.Tpo .deps/zero_copy_stream_impl_lite.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT structurally_valid.lo -MD -MP -MF .deps/structurally_valid.Tpo -c -o structurally_valid.lo `test -f 'google/protobuf/stubs/structurally_valid.cc' || echo './'`google/protobuf/stubs/structurally_valid.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT structurally_valid.lo -MD -MP -MF .deps/structurally_valid.Tpo -c google/protobuf/stubs/structurally_valid.cc  -fPIC -DPIC -o structurally_valid.o
mv -f .deps/substitute.Tpo .deps/substitute.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.lo -MD -MP -MF .deps/descriptor.Tpo -c -o descriptor.lo `test -f 'google/protobuf/descriptor.cc' || echo './'`google/protobuf/descriptor.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.lo -MD -MP -MF .deps/descriptor.Tpo -c google/protobuf/descriptor.cc  -fPIC -DPIC -o descriptor.o
mv -f .deps/structurally_valid.Tpo .deps/structurally_valid.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.pb.lo -MD -MP -MF .deps/descriptor.pb.Tpo -c -o descriptor.pb.lo `test -f 'google/protobuf/descriptor.pb.cc' || echo './'`google/protobuf/descriptor.pb.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor.pb.lo -MD -MP -MF .deps/descriptor.pb.Tpo -c google/protobuf/descriptor.pb.cc  -fPIC -DPIC -o descriptor.pb.o
mv -f .deps/strutil.Tpo .deps/strutil.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor_database.lo -MD -MP -MF .deps/descriptor_database.Tpo -c -o descriptor_database.lo `test -f 'google/protobuf/descriptor_database.cc' || echo './'`google/protobuf/descriptor_database.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT descriptor_database.lo -MD -MP -MF .deps/descriptor_database.Tpo -c google/protobuf/descriptor_database.cc  -fPIC -DPIC -o descriptor_database.o
mv -f .deps/descriptor_database.Tpo .deps/descriptor_database.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT dynamic_message.lo -MD -MP -MF .deps/dynamic_message.Tpo -c -o dynamic_message.lo `test -f 'google/protobuf/dynamic_message.cc' || echo './'`google/protobuf/dynamic_message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT dynamic_message.lo -MD -MP -MF .deps/dynamic_message.Tpo -c google/protobuf/dynamic_message.cc  -fPIC -DPIC -o dynamic_message.o
mv -f .deps/descriptor.pb.Tpo .deps/descriptor.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set_heavy.lo -MD -MP -MF .deps/extension_set_heavy.Tpo -c -o extension_set_heavy.lo `test -f 'google/protobuf/extension_set_heavy.cc' || echo './'`google/protobuf/extension_set_heavy.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT extension_set_heavy.lo -MD -MP -MF .deps/extension_set_heavy.Tpo -c google/protobuf/extension_set_heavy.cc  -fPIC -DPIC -o extension_set_heavy.o
mv -f .deps/dynamic_message.Tpo .deps/dynamic_message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_reflection.lo -MD -MP -MF .deps/generated_message_reflection.Tpo -c -o generated_message_reflection.lo `test -f 'google/protobuf/generated_message_reflection.cc' || echo './'`google/protobuf/generated_message_reflection.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT generated_message_reflection.lo -MD -MP -MF .deps/generated_message_reflection.Tpo -c google/protobuf/generated_message_reflection.cc  -fPIC -DPIC -o generated_message_reflection.o
mv -f .deps/extension_set_heavy.Tpo .deps/extension_set_heavy.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message.lo -MD -MP -MF .deps/message.Tpo -c -o message.lo `test -f 'google/protobuf/message.cc' || echo './'`google/protobuf/message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT message.lo -MD -MP -MF .deps/message.Tpo -c google/protobuf/message.cc  -fPIC -DPIC -o message.o
mv -f .deps/descriptor.Tpo .deps/descriptor.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT reflection_ops.lo -MD -MP -MF .deps/reflection_ops.Tpo -c -o reflection_ops.lo `test -f 'google/protobuf/reflection_ops.cc' || echo './'`google/protobuf/reflection_ops.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT reflection_ops.lo -MD -MP -MF .deps/reflection_ops.Tpo -c google/protobuf/reflection_ops.cc  -fPIC -DPIC -o reflection_ops.o
mv -f .deps/generated_message_reflection.Tpo .deps/generated_message_reflection.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT service.lo -MD -MP -MF .deps/service.Tpo -c -o service.lo `test -f 'google/protobuf/service.cc' || echo './'`google/protobuf/service.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT service.lo -MD -MP -MF .deps/service.Tpo -c google/protobuf/service.cc  -fPIC -DPIC -o service.o
mv -f .deps/service.Tpo .deps/service.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT text_format.lo -MD -MP -MF .deps/text_format.Tpo -c -o text_format.lo `test -f 'google/protobuf/text_format.cc' || echo './'`google/protobuf/text_format.cc
mv -f .deps/reflection_ops.Tpo .deps/reflection_ops.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT unknown_field_set.lo -MD -MP -MF .deps/unknown_field_set.Tpo -c -o unknown_field_set.lo `test -f 'google/protobuf/unknown_field_set.cc' || echo './'`google/protobuf/unknown_field_set.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT text_format.lo -MD -MP -MF .deps/text_format.Tpo -c google/protobuf/text_format.cc  -fPIC -DPIC -o text_format.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT unknown_field_set.lo -MD -MP -MF .deps/unknown_field_set.Tpo -c google/protobuf/unknown_field_set.cc  -fPIC -DPIC -o unknown_field_set.o
mv -f .deps/message.Tpo .deps/message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format.lo -MD -MP -MF .deps/wire_format.Tpo -c -o wire_format.lo `test -f 'google/protobuf/wire_format.cc' || echo './'`google/protobuf/wire_format.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT wire_format.lo -MD -MP -MF .deps/wire_format.Tpo -c google/protobuf/wire_format.cc  -fPIC -DPIC -o wire_format.o
mv -f .deps/unknown_field_set.Tpo .deps/unknown_field_set.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT gzip_stream.lo -MD -MP -MF .deps/gzip_stream.Tpo -c -o gzip_stream.lo `test -f 'google/protobuf/io/gzip_stream.cc' || echo './'`google/protobuf/io/gzip_stream.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT gzip_stream.lo -MD -MP -MF .deps/gzip_stream.Tpo -c google/protobuf/io/gzip_stream.cc  -fPIC -DPIC -o gzip_stream.o
mv -f .deps/gzip_stream.Tpo .deps/gzip_stream.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT printer.lo -MD -MP -MF .deps/printer.Tpo -c -o printer.lo `test -f 'google/protobuf/io/printer.cc' || echo './'`google/protobuf/io/printer.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT printer.lo -MD -MP -MF .deps/printer.Tpo -c google/protobuf/io/printer.cc  -fPIC -DPIC -o printer.o
mv -f .deps/wire_format.Tpo .deps/wire_format.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT tokenizer.lo -MD -MP -MF .deps/tokenizer.Tpo -c -o tokenizer.lo `test -f 'google/protobuf/io/tokenizer.cc' || echo './'`google/protobuf/io/tokenizer.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT tokenizer.lo -MD -MP -MF .deps/tokenizer.Tpo -c google/protobuf/io/tokenizer.cc  -fPIC -DPIC -o tokenizer.o
mv -f .deps/text_format.Tpo .deps/text_format.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl.lo -MD -MP -MF .deps/zero_copy_stream_impl.Tpo -c -o zero_copy_stream_impl.lo `test -f 'google/protobuf/io/zero_copy_stream_impl.cc' || echo './'`google/protobuf/io/zero_copy_stream_impl.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zero_copy_stream_impl.lo -MD -MP -MF .deps/zero_copy_stream_impl.Tpo -c google/protobuf/io/zero_copy_stream_impl.cc  -fPIC -DPIC -o zero_copy_stream_impl.o
mv -f .deps/printer.Tpo .deps/printer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT importer.lo -MD -MP -MF .deps/importer.Tpo -c -o importer.lo `test -f 'google/protobuf/compiler/importer.cc' || echo './'`google/protobuf/compiler/importer.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT importer.lo -MD -MP -MF .deps/importer.Tpo -c google/protobuf/compiler/importer.cc  -fPIC -DPIC -o importer.o
mv -f .deps/tokenizer.Tpo .deps/tokenizer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT parser.lo -MD -MP -MF .deps/parser.Tpo -c -o parser.lo `test -f 'google/protobuf/compiler/parser.cc' || echo './'`google/protobuf/compiler/parser.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT parser.lo -MD -MP -MF .deps/parser.Tpo -c google/protobuf/compiler/parser.cc  -fPIC -DPIC -o parser.o
mv -f .deps/zero_copy_stream_impl.Tpo .deps/zero_copy_stream_impl.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT code_generator.lo -MD -MP -MF .deps/code_generator.Tpo -c -o code_generator.lo `test -f 'google/protobuf/compiler/code_generator.cc' || echo './'`google/protobuf/compiler/code_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT code_generator.lo -MD -MP -MF .deps/code_generator.Tpo -c google/protobuf/compiler/code_generator.cc  -fPIC -DPIC -o code_generator.o
mv -f .deps/code_generator.Tpo .deps/code_generator.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT command_line_interface.lo -MD -MP -MF .deps/command_line_interface.Tpo -c -o command_line_interface.lo `test -f 'google/protobuf/compiler/command_line_interface.cc' || echo './'`google/protobuf/compiler/command_line_interface.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT command_line_interface.lo -MD -MP -MF .deps/command_line_interface.Tpo -c google/protobuf/compiler/command_line_interface.cc  -fPIC -DPIC -o command_line_interface.o
mv -f .deps/importer.Tpo .deps/importer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.lo -MD -MP -MF .deps/plugin.Tpo -c -o plugin.lo `test -f 'google/protobuf/compiler/plugin.cc' || echo './'`google/protobuf/compiler/plugin.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.lo -MD -MP -MF .deps/plugin.Tpo -c google/protobuf/compiler/plugin.cc  -fPIC -DPIC -o plugin.o
mv -f .deps/parser.Tpo .deps/parser.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.pb.lo -MD -MP -MF .deps/plugin.pb.Tpo -c -o plugin.pb.lo `test -f 'google/protobuf/compiler/plugin.pb.cc' || echo './'`google/protobuf/compiler/plugin.pb.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT plugin.pb.lo -MD -MP -MF .deps/plugin.pb.Tpo -c google/protobuf/compiler/plugin.pb.cc  -fPIC -DPIC -o plugin.pb.o
mv -f .deps/plugin.Tpo .deps/plugin.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT subprocess.lo -MD -MP -MF .deps/subprocess.Tpo -c -o subprocess.lo `test -f 'google/protobuf/compiler/subprocess.cc' || echo './'`google/protobuf/compiler/subprocess.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT subprocess.lo -MD -MP -MF .deps/subprocess.Tpo -c google/protobuf/compiler/subprocess.cc  -fPIC -DPIC -o subprocess.o
mv -f .deps/subprocess.Tpo .deps/subprocess.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zip_writer.lo -MD -MP -MF .deps/zip_writer.Tpo -c -o zip_writer.lo `test -f 'google/protobuf/compiler/zip_writer.cc' || echo './'`google/protobuf/compiler/zip_writer.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT zip_writer.lo -MD -MP -MF .deps/zip_writer.Tpo -c google/protobuf/compiler/zip_writer.cc  -fPIC -DPIC -o zip_writer.o
mv -f .deps/command_line_interface.Tpo .deps/command_line_interface.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum.lo -MD -MP -MF .deps/cpp_enum.Tpo -c -o cpp_enum.lo `test -f 'google/protobuf/compiler/cpp/cpp_enum.cc' || echo './'`google/protobuf/compiler/cpp/cpp_enum.cc
mv -f .deps/plugin.pb.Tpo .deps/plugin.pb.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum.lo -MD -MP -MF .deps/cpp_enum.Tpo -c google/protobuf/compiler/cpp/cpp_enum.cc  -fPIC -DPIC -o cpp_enum.o
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum_field.lo -MD -MP -MF .deps/cpp_enum_field.Tpo -c -o cpp_enum_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_enum_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_enum_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_enum_field.lo -MD -MP -MF .deps/cpp_enum_field.Tpo -c google/protobuf/compiler/cpp/cpp_enum_field.cc  -fPIC -DPIC -o cpp_enum_field.o
mv -f .deps/zip_writer.Tpo .deps/zip_writer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_extension.lo -MD -MP -MF .deps/cpp_extension.Tpo -c -o cpp_extension.lo `test -f 'google/protobuf/compiler/cpp/cpp_extension.cc' || echo './'`google/protobuf/compiler/cpp/cpp_extension.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_extension.lo -MD -MP -MF .deps/cpp_extension.Tpo -c google/protobuf/compiler/cpp/cpp_extension.cc  -fPIC -DPIC -o cpp_extension.o
mv -f .deps/cpp_enum_field.Tpo .deps/cpp_enum_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_field.lo -MD -MP -MF .deps/cpp_field.Tpo -c -o cpp_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_field.lo -MD -MP -MF .deps/cpp_field.Tpo -c google/protobuf/compiler/cpp/cpp_field.cc  -fPIC -DPIC -o cpp_field.o
mv -f .deps/cpp_enum.Tpo .deps/cpp_enum.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_file.lo -MD -MP -MF .deps/cpp_file.Tpo -c -o cpp_file.lo `test -f 'google/protobuf/compiler/cpp/cpp_file.cc' || echo './'`google/protobuf/compiler/cpp/cpp_file.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_file.lo -MD -MP -MF .deps/cpp_file.Tpo -c google/protobuf/compiler/cpp/cpp_file.cc  -fPIC -DPIC -o cpp_file.o
mv -f .deps/cpp_extension.Tpo .deps/cpp_extension.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_generator.lo -MD -MP -MF .deps/cpp_generator.Tpo -c -o cpp_generator.lo `test -f 'google/protobuf/compiler/cpp/cpp_generator.cc' || echo './'`google/protobuf/compiler/cpp/cpp_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_generator.lo -MD -MP -MF .deps/cpp_generator.Tpo -c google/protobuf/compiler/cpp/cpp_generator.cc  -fPIC -DPIC -o cpp_generator.o
mv -f .deps/cpp_generator.Tpo .deps/cpp_generator.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_helpers.lo -MD -MP -MF .deps/cpp_helpers.Tpo -c -o cpp_helpers.lo `test -f 'google/protobuf/compiler/cpp/cpp_helpers.cc' || echo './'`google/protobuf/compiler/cpp/cpp_helpers.cc
mv -f .deps/cpp_file.Tpo .deps/cpp_file.Plo
mv -f .deps/cpp_field.Tpo .deps/cpp_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message.lo -MD -MP -MF .deps/cpp_message.Tpo -c -o cpp_message.lo `test -f 'google/protobuf/compiler/cpp/cpp_message.cc' || echo './'`google/protobuf/compiler/cpp/cpp_message.cc
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message_field.lo -MD -MP -MF .deps/cpp_message_field.Tpo -c -o cpp_message_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_message_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_message_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_helpers.lo -MD -MP -MF .deps/cpp_helpers.Tpo -c google/protobuf/compiler/cpp/cpp_helpers.cc  -fPIC -DPIC -o cpp_helpers.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message.lo -MD -MP -MF .deps/cpp_message.Tpo -c google/protobuf/compiler/cpp/cpp_message.cc  -fPIC -DPIC -o cpp_message.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_message_field.lo -MD -MP -MF .deps/cpp_message_field.Tpo -c google/protobuf/compiler/cpp/cpp_message_field.cc  -fPIC -DPIC -o cpp_message_field.o
mv -f .deps/cpp_message_field.Tpo .deps/cpp_message_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_primitive_field.lo -MD -MP -MF .deps/cpp_primitive_field.Tpo -c -o cpp_primitive_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_primitive_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_primitive_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_primitive_field.lo -MD -MP -MF .deps/cpp_primitive_field.Tpo -c google/protobuf/compiler/cpp/cpp_primitive_field.cc  -fPIC -DPIC -o cpp_primitive_field.o
mv -f .deps/cpp_helpers.Tpo .deps/cpp_helpers.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_service.lo -MD -MP -MF .deps/cpp_service.Tpo -c -o cpp_service.lo `test -f 'google/protobuf/compiler/cpp/cpp_service.cc' || echo './'`google/protobuf/compiler/cpp/cpp_service.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_service.lo -MD -MP -MF .deps/cpp_service.Tpo -c google/protobuf/compiler/cpp/cpp_service.cc  -fPIC -DPIC -o cpp_service.o
mv -f .deps/cpp_message.Tpo .deps/cpp_message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_string_field.lo -MD -MP -MF .deps/cpp_string_field.Tpo -c -o cpp_string_field.lo `test -f 'google/protobuf/compiler/cpp/cpp_string_field.cc' || echo './'`google/protobuf/compiler/cpp/cpp_string_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT cpp_string_field.lo -MD -MP -MF .deps/cpp_string_field.Tpo -c google/protobuf/compiler/cpp/cpp_string_field.cc  -fPIC -DPIC -o cpp_string_field.o
mv -f .deps/cpp_primitive_field.Tpo .deps/cpp_primitive_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum.lo -MD -MP -MF .deps/java_enum.Tpo -c -o java_enum.lo `test -f 'google/protobuf/compiler/java/java_enum.cc' || echo './'`google/protobuf/compiler/java/java_enum.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum.lo -MD -MP -MF .deps/java_enum.Tpo -c google/protobuf/compiler/java/java_enum.cc  -fPIC -DPIC -o java_enum.o
mv -f .deps/cpp_service.Tpo .deps/cpp_service.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum_field.lo -MD -MP -MF .deps/java_enum_field.Tpo -c -o java_enum_field.lo `test -f 'google/protobuf/compiler/java/java_enum_field.cc' || echo './'`google/protobuf/compiler/java/java_enum_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_enum_field.lo -MD -MP -MF .deps/java_enum_field.Tpo -c google/protobuf/compiler/java/java_enum_field.cc  -fPIC -DPIC -o java_enum_field.o
mv -f .deps/cpp_string_field.Tpo .deps/cpp_string_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_extension.lo -MD -MP -MF .deps/java_extension.Tpo -c -o java_extension.lo `test -f 'google/protobuf/compiler/java/java_extension.cc' || echo './'`google/protobuf/compiler/java/java_extension.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_extension.lo -MD -MP -MF .deps/java_extension.Tpo -c google/protobuf/compiler/java/java_extension.cc  -fPIC -DPIC -o java_extension.o
mv -f .deps/java_enum.Tpo .deps/java_enum.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_field.lo -MD -MP -MF .deps/java_field.Tpo -c -o java_field.lo `test -f 'google/protobuf/compiler/java/java_field.cc' || echo './'`google/protobuf/compiler/java/java_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_field.lo -MD -MP -MF .deps/java_field.Tpo -c google/protobuf/compiler/java/java_field.cc  -fPIC -DPIC -o java_field.o
mv -f .deps/java_enum_field.Tpo .deps/java_enum_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_file.lo -MD -MP -MF .deps/java_file.Tpo -c -o java_file.lo `test -f 'google/protobuf/compiler/java/java_file.cc' || echo './'`google/protobuf/compiler/java/java_file.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_file.lo -MD -MP -MF .deps/java_file.Tpo -c google/protobuf/compiler/java/java_file.cc  -fPIC -DPIC -o java_file.o
mv -f .deps/java_extension.Tpo .deps/java_extension.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_generator.lo -MD -MP -MF .deps/java_generator.Tpo -c -o java_generator.lo `test -f 'google/protobuf/compiler/java/java_generator.cc' || echo './'`google/protobuf/compiler/java/java_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_generator.lo -MD -MP -MF .deps/java_generator.Tpo -c google/protobuf/compiler/java/java_generator.cc  -fPIC -DPIC -o java_generator.o
mv -f .deps/java_field.Tpo .deps/java_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_helpers.lo -MD -MP -MF .deps/java_helpers.Tpo -c -o java_helpers.lo `test -f 'google/protobuf/compiler/java/java_helpers.cc' || echo './'`google/protobuf/compiler/java/java_helpers.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_helpers.lo -MD -MP -MF .deps/java_helpers.Tpo -c google/protobuf/compiler/java/java_helpers.cc  -fPIC -DPIC -o java_helpers.o
mv -f .deps/java_file.Tpo .deps/java_file.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message.lo -MD -MP -MF .deps/java_message.Tpo -c -o java_message.lo `test -f 'google/protobuf/compiler/java/java_message.cc' || echo './'`google/protobuf/compiler/java/java_message.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message.lo -MD -MP -MF .deps/java_message.Tpo -c google/protobuf/compiler/java/java_message.cc  -fPIC -DPIC -o java_message.o
mv -f .deps/java_generator.Tpo .deps/java_generator.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message_field.lo -MD -MP -MF .deps/java_message_field.Tpo -c -o java_message_field.lo `test -f 'google/protobuf/compiler/java/java_message_field.cc' || echo './'`google/protobuf/compiler/java/java_message_field.cc
mv -f .deps/java_helpers.Tpo .deps/java_helpers.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_primitive_field.lo -MD -MP -MF .deps/java_primitive_field.Tpo -c -o java_primitive_field.lo `test -f 'google/protobuf/compiler/java/java_primitive_field.cc' || echo './'`google/protobuf/compiler/java/java_primitive_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_message_field.lo -MD -MP -MF .deps/java_message_field.Tpo -c google/protobuf/compiler/java/java_message_field.cc  -fPIC -DPIC -o java_message_field.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_primitive_field.lo -MD -MP -MF .deps/java_primitive_field.Tpo -c google/protobuf/compiler/java/java_primitive_field.cc  -fPIC -DPIC -o java_primitive_field.o
mv -f .deps/java_message.Tpo .deps/java_message.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_service.lo -MD -MP -MF .deps/java_service.Tpo -c -o java_service.lo `test -f 'google/protobuf/compiler/java/java_service.cc' || echo './'`google/protobuf/compiler/java/java_service.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_service.lo -MD -MP -MF .deps/java_service.Tpo -c google/protobuf/compiler/java/java_service.cc  -fPIC -DPIC -o java_service.o
mv -f .deps/java_message_field.Tpo .deps/java_message_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_string_field.lo -MD -MP -MF .deps/java_string_field.Tpo -c -o java_string_field.lo `test -f 'google/protobuf/compiler/java/java_string_field.cc' || echo './'`google/protobuf/compiler/java/java_string_field.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_string_field.lo -MD -MP -MF .deps/java_string_field.Tpo -c google/protobuf/compiler/java/java_string_field.cc  -fPIC -DPIC -o java_string_field.o
mv -f .deps/java_primitive_field.Tpo .deps/java_primitive_field.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_doc_comment.lo -MD -MP -MF .deps/java_doc_comment.Tpo -c -o java_doc_comment.lo `test -f 'google/protobuf/compiler/java/java_doc_comment.cc' || echo './'`google/protobuf/compiler/java/java_doc_comment.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT java_doc_comment.lo -MD -MP -MF .deps/java_doc_comment.Tpo -c google/protobuf/compiler/java/java_doc_comment.cc  -fPIC -DPIC -o java_doc_comment.o
mv -f .deps/java_doc_comment.Tpo .deps/java_doc_comment.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..    -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT python_generator.lo -MD -MP -MF .deps/python_generator.Tpo -c -o python_generator.lo `test -f 'google/protobuf/compiler/python/python_generator.cc' || echo './'`google/protobuf/compiler/python/python_generator.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT python_generator.lo -MD -MP -MF .deps/python_generator.Tpo -c google/protobuf/compiler/python/python_generator.cc  -fPIC -DPIC -o python_generator.o
mv -f .deps/java_service.Tpo .deps/java_service.Plo
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -version-info 8:0:0 -export-dynamic -no-undefined  -o libprotobuf.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib atomicops_internals_x86_gcc.lo atomicops_internals_x86_msvc.lo common.lo once.lo stringprintf.lo extension_set.lo generated_message_util.lo message_lite.lo repeated_field.lo wire_format_lite.lo coded_stream.lo zero_copy_stream.lo zero_copy_stream_impl_lite.lo strutil.lo substitute.lo structurally_valid.lo descriptor.lo descriptor.pb.lo descriptor_database.lo dynamic_message.lo extension_set_heavy.lo generated_message_reflection.lo message.lo reflection_ops.lo service.lo text_format.lo unknown_field_set.lo wire_format.lo gzip_stream.lo printer.lo tokenizer.lo zero_copy_stream_impl.lo importer.lo parser.lo -lpthread -lz 
libtool: link: ar cru .libs/libprotobuf.a  atomicops_internals_x86_gcc.o atomicops_internals_x86_msvc.o common.o once.o stringprintf.o extension_set.o generated_message_util.o message_lite.o repeated_field.o wire_format_lite.o coded_stream.o zero_copy_stream.o zero_copy_stream_impl_lite.o strutil.o substitute.o structurally_valid.o descriptor.o descriptor.pb.o descriptor_database.o dynamic_message.o extension_set_heavy.o generated_message_reflection.o message.o reflection_ops.o service.o text_format.o unknown_field_set.o wire_format.o gzip_stream.o printer.o tokenizer.o zero_copy_stream_impl.o importer.o parser.o
mv -f .deps/java_string_field.Tpo .deps/java_string_field.Plo
libtool: link: ranlib .libs/libprotobuf.a
libtool: link: ( cd ".libs" && rm -f "libprotobuf.la" && ln -s "../libprotobuf.la" "libprotobuf.la" )
mv -f .deps/python_generator.Tpo .deps/python_generator.Plo
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -version-info 8:0:0 -export-dynamic -no-undefined  -o libprotoc.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib code_generator.lo command_line_interface.lo plugin.lo plugin.pb.lo subprocess.lo zip_writer.lo cpp_enum.lo cpp_enum_field.lo cpp_extension.lo cpp_field.lo cpp_file.lo cpp_generator.lo cpp_helpers.lo cpp_message.lo cpp_message_field.lo cpp_primitive_field.lo cpp_service.lo cpp_string_field.lo java_enum.lo java_enum_field.lo java_extension.lo java_field.lo java_file.lo java_generator.lo java_helpers.lo java_message.lo java_message_field.lo java_primitive_field.lo java_service.lo java_string_field.lo java_doc_comment.lo python_generator.lo -lpthread libprotobuf.la -lz 
libtool: link: ar cru .libs/libprotoc.a  code_generator.o command_line_interface.o plugin.o plugin.pb.o subprocess.o zip_writer.o cpp_enum.o cpp_enum_field.o cpp_extension.o cpp_field.o cpp_file.o cpp_generator.o cpp_helpers.o cpp_message.o cpp_message_field.o cpp_primitive_field.o cpp_service.o cpp_string_field.o java_enum.o java_enum_field.o java_extension.o java_field.o java_file.o java_generator.o java_helpers.o java_message.o java_message_field.o java_primitive_field.o java_service.o java_string_field.o java_doc_comment.o python_generator.o
libtool: link: ranlib .libs/libprotoc.a
libtool: link: ( cd ".libs" && rm -f "libprotoc.la" && ln -s "../libprotoc.la" "libprotoc.la" )
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread  -o protoc main.o -lpthread libprotobuf.la libprotoc.la -lz 
libtool: link: g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -pthread -o protoc main.o  ./.libs/libprotobuf.a ./.libs/libprotoc.a /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/.libs/libprotobuf.a -lpthread -lz -pthread
oldpwd=`pwd` && ( cd . && $oldpwd/protoc -I. --cpp_out=$oldpwd google/protobuf/unittest.proto google/protobuf/unittest_empty.proto google/protobuf/unittest_import.proto google/protobuf/unittest_import_public.proto google/protobuf/unittest_mset.proto google/protobuf/unittest_optimize_for.proto google/protobuf/unittest_embed_optimize_for.proto google/protobuf/unittest_custom_options.proto google/protobuf/unittest_lite.proto google/protobuf/unittest_import_lite.proto google/protobuf/unittest_import_public_lite.proto google/protobuf/unittest_lite_imports_nonlite.proto google/protobuf/unittest_no_generic_services.proto google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto )
touch unittest_proto_middleman
make  all-am
make[11]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
/bin/bash ../libtool --tag=CXX   --mode=link g++ -pthread -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare  -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -version-info 8:0:0 -export-dynamic -no-undefined  -o libprotobuf-lite.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib atomicops_internals_x86_gcc.lo atomicops_internals_x86_msvc.lo common.lo once.lo stringprintf.lo extension_set.lo generated_message_util.lo message_lite.lo repeated_field.lo wire_format_lite.lo coded_stream.lo zero_copy_stream.lo zero_copy_stream_impl_lite.lo -lpthread -lz 
libtool: link: ar cru .libs/libprotobuf-lite.a  atomicops_internals_x86_gcc.o atomicops_internals_x86_msvc.o common.o once.o stringprintf.o extension_set.o generated_message_util.o message_lite.o repeated_field.o wire_format_lite.o coded_stream.o zero_copy_stream.o zero_copy_stream_impl_lite.o
libtool: link: ranlib .libs/libprotobuf-lite.a
libtool: link: ( cd ".libs" && rm -f "libprotobuf-lite.la" && ln -s "../libprotobuf-lite.la" "libprotobuf-lite.la" )
make[11]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
make[10]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src'
make[9]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
make[8]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/protobuf-2.5.0'
touch protobuf-2.5.0-build-stamp
make[7]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
make[6]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty'
Making all in .
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-clock.lo -MD -MP -MF .deps/libprocess_la-clock.Tpo -c -o libprocess_la-clock.lo `test -f 'src/clock.cpp' || echo '../../../3rdparty/libprocess/'`src/clock.cpp
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-help.lo -MD -MP -MF .deps/libprocess_la-help.Tpo -c -o libprocess_la-help.lo `test -f 'src/help.cpp' || echo '../../../3rdparty/libprocess/'`src/help.cpp
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-http.lo -MD -MP -MF .deps/libprocess_la-http.Tpo -c -o libprocess_la-http.lo `test -f 'src/http.cpp' || echo '../../../3rdparty/libprocess/'`src/http.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-clock.lo -MD -MP -MF .deps/libprocess_la-clock.Tpo -c ../../../3rdparty/libprocess/src/clock.cpp  -fPIC -DPIC -o libprocess_la-clock.o
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-http.lo -MD -MP -MF .deps/libprocess_la-http.Tpo -c ../../../3rdparty/libprocess/src/http.cpp  -fPIC -DPIC -o libprocess_la-http.o
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-help.lo -MD -MP -MF .deps/libprocess_la-help.Tpo -c ../../../3rdparty/libprocess/src/help.cpp  -fPIC -DPIC -o libprocess_la-help.o
mv -f .deps/libprocess_la-help.Tpo .deps/libprocess_la-help.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-io.lo -MD -MP -MF .deps/libprocess_la-io.Tpo -c -o libprocess_la-io.lo `test -f 'src/io.cpp' || echo '../../../3rdparty/libprocess/'`src/io.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-io.lo -MD -MP -MF .deps/libprocess_la-io.Tpo -c ../../../3rdparty/libprocess/src/io.cpp  -fPIC -DPIC -o libprocess_la-io.o
mv -f .deps/libprocess_la-clock.Tpo .deps/libprocess_la-clock.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-latch.lo -MD -MP -MF .deps/libprocess_la-latch.Tpo -c -o libprocess_la-latch.lo `test -f 'src/latch.cpp' || echo '../../../3rdparty/libprocess/'`src/latch.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-latch.lo -MD -MP -MF .deps/libprocess_la-latch.Tpo -c ../../../3rdparty/libprocess/src/latch.cpp  -fPIC -DPIC -o libprocess_la-latch.o
mv -f .deps/libprocess_la-http.Tpo .deps/libprocess_la-http.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-metrics.lo -MD -MP -MF .deps/libprocess_la-metrics.Tpo -c -o libprocess_la-metrics.lo `test -f 'src/metrics/metrics.cpp' || echo '../../../3rdparty/libprocess/'`src/metrics/metrics.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-metrics.lo -MD -MP -MF .deps/libprocess_la-metrics.Tpo -c ../../../3rdparty/libprocess/src/metrics/metrics.cpp  -fPIC -DPIC -o libprocess_la-metrics.o
mv -f .deps/libprocess_la-latch.Tpo .deps/libprocess_la-latch.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-pid.lo -MD -MP -MF .deps/libprocess_la-pid.Tpo -c -o libprocess_la-pid.lo `test -f 'src/pid.cpp' || echo '../../../3rdparty/libprocess/'`src/pid.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-pid.lo -MD -MP -MF .deps/libprocess_la-pid.Tpo -c ../../../3rdparty/libprocess/src/pid.cpp  -fPIC -DPIC -o libprocess_la-pid.o
mv -f .deps/libprocess_la-io.Tpo .deps/libprocess_la-io.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-poll_socket.lo -MD -MP -MF .deps/libprocess_la-poll_socket.Tpo -c -o libprocess_la-poll_socket.lo `test -f 'src/poll_socket.cpp' || echo '../../../3rdparty/libprocess/'`src/poll_socket.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-poll_socket.lo -MD -MP -MF .deps/libprocess_la-poll_socket.Tpo -c ../../../3rdparty/libprocess/src/poll_socket.cpp  -fPIC -DPIC -o libprocess_la-poll_socket.o
mv -f .deps/libprocess_la-pid.Tpo .deps/libprocess_la-pid.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-process.lo -MD -MP -MF .deps/libprocess_la-process.Tpo -c -o libprocess_la-process.lo `test -f 'src/process.cpp' || echo '../../../3rdparty/libprocess/'`src/process.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-process.lo -MD -MP -MF .deps/libprocess_la-process.Tpo -c ../../../3rdparty/libprocess/src/process.cpp  -fPIC -DPIC -o libprocess_la-process.o
mv -f .deps/libprocess_la-poll_socket.Tpo .deps/libprocess_la-poll_socket.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c -o libprocess_la-reap.lo `test -f 'src/reap.cpp' || echo '../../../3rdparty/libprocess/'`src/reap.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-reap.lo -MD -MP -MF .deps/libprocess_la-reap.Tpo -c ../../../3rdparty/libprocess/src/reap.cpp  -fPIC -DPIC -o libprocess_la-reap.o
mv -f .deps/libprocess_la-metrics.Tpo .deps/libprocess_la-metrics.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-socket.lo -MD -MP -MF .deps/libprocess_la-socket.Tpo -c -o libprocess_la-socket.lo `test -f 'src/socket.cpp' || echo '../../../3rdparty/libprocess/'`src/socket.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-socket.lo -MD -MP -MF .deps/libprocess_la-socket.Tpo -c ../../../3rdparty/libprocess/src/socket.cpp  -fPIC -DPIC -o libprocess_la-socket.o
mv -f .deps/libprocess_la-reap.Tpo .deps/libprocess_la-reap.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-subprocess.lo -MD -MP -MF .deps/libprocess_la-subprocess.Tpo -c -o libprocess_la-subprocess.lo `test -f 'src/subprocess.cpp' || echo '../../../3rdparty/libprocess/'`src/subprocess.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-subprocess.lo -MD -MP -MF .deps/libprocess_la-subprocess.Tpo -c ../../../3rdparty/libprocess/src/subprocess.cpp  -fPIC -DPIC -o libprocess_la-subprocess.o
mv -f .deps/libprocess_la-socket.Tpo .deps/libprocess_la-socket.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-timeseries.lo -MD -MP -MF .deps/libprocess_la-timeseries.Tpo -c -o libprocess_la-timeseries.lo `test -f 'src/timeseries.cpp' || echo '../../../3rdparty/libprocess/'`src/timeseries.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-timeseries.lo -MD -MP -MF .deps/libprocess_la-timeseries.Tpo -c ../../../3rdparty/libprocess/src/timeseries.cpp  -fPIC -DPIC -o libprocess_la-timeseries.o
mv -f .deps/libprocess_la-timeseries.Tpo .deps/libprocess_la-timeseries.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev.lo -MD -MP -MF .deps/libprocess_la-libev.Tpo -c -o libprocess_la-libev.lo `test -f 'src/libev.cpp' || echo '../../../3rdparty/libprocess/'`src/libev.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev.lo -MD -MP -MF .deps/libprocess_la-libev.Tpo -c ../../../3rdparty/libprocess/src/libev.cpp  -fPIC -DPIC -o libprocess_la-libev.o
mv -f .deps/libprocess_la-subprocess.Tpo .deps/libprocess_la-subprocess.Plo
/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" -DPACKAGE_STRING=\"libprocess\ 0.0.1\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess  -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734  -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a  -I/usr/include/subversion-1 -I/usr/include
 /apr-1 -I/usr/include/apr-1.0   -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev_poll.lo -MD -MP -MF .deps/libprocess_la-libev_poll.Tpo -c -o libprocess_la-libev_poll.lo `test -f 'src/libev_poll.cpp' || echo '../../../3rdparty/libprocess/'`src/libev_poll.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"libprocess\" -DPACKAGE_TARNAME=\"libprocess\" -DPACKAGE_VERSION=\"0.0.1\" "-DPACKAGE_STRING=\"libprocess 0.0.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libprocess\" -DVERSION=\"0.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBCURL=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBDL=1 -I. -I../../../3rdparty/libprocess -I../../../3rdparty/libprocess/include -I../../../3rdparty/libprocess/3rdparty/stout/include -I3rdparty/boost-1.53.0 -I3rdparty/libev-4.15 -I3rdparty/picojson-4f93734 -I3rdparty/glog-0.3.3/src -I3rdparty/ry-http-parser-1c3624a -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -
 g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libprocess_la-libev_poll.lo -MD -MP -MF .deps/libprocess_la-libev_poll.Tpo -c ../../../3rdparty/libprocess/src/libev_poll.cpp  -fPIC -DPIC -o libprocess_la-libev_poll.o
mv -f .deps/libprocess_la-libev.Tpo .deps/libprocess_la-libev.Plo
mv -f .deps/libprocess_la-process.Tpo .deps/libprocess_la-process.Plo
mv -f .deps/libprocess_la-libev_poll.Tpo .deps/libprocess_la-libev_poll.Plo
/bin/bash ./libtool  --tag=CXX   --mode=link g++  -g1 -O0 -Wno-unused-local-typedefs -std=c++11   -o libprocess.la  libprocess_la-clock.lo libprocess_la-help.lo libprocess_la-http.lo libprocess_la-io.lo libprocess_la-latch.lo libprocess_la-metrics.lo libprocess_la-pid.lo libprocess_la-poll_socket.lo libprocess_la-process.lo libprocess_la-reap.lo libprocess_la-socket.lo libprocess_la-subprocess.lo libprocess_la-timeseries.lo  libprocess_la-libev.lo libprocess_la-libev_poll.lo 3rdparty/glog-0.3.3/libglog.la 3rdparty/libry_http_parser.la 3rdparty/libev-4.15/libev.la  -ldl -lz -lcurl -lsvn_delta-1 -lsvn_subr-1 -lapr-1  -lrt
libtool: link: (cd .libs/libprocess.lax/libry_http_parser.a && ar x "/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/3rdparty/.libs/libry_http_parser.a")
libtool: link: ar cru .libs/libprocess.a  libprocess_la-clock.o libprocess_la-help.o libprocess_la-http.o libprocess_la-io.o libprocess_la-latch.o libprocess_la-metrics.o libprocess_la-pid.o libprocess_la-poll_socket.o libprocess_la-process.o libprocess_la-reap.o libprocess_la-socket.o libprocess_la-subprocess.o libprocess_la-timeseries.o libprocess_la-libev.o libprocess_la-libev_poll.o  .libs/libprocess.lax/libry_http_parser.a/libry_http_parser_la-http_parser.o 
libtool: link: ranlib .libs/libprocess.a
libtool: link: rm -fr .libs/libprocess.lax
libtool: link: ( cd ".libs" && rm -f "libprocess.la" && ln -s "../libprocess.la" "libprocess.la" )
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
Making all in include
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/include'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess/include'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/libprocess'
make[4]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
gzip -d -c ../../3rdparty/distribute-0.6.26.tar.gz | tar xf -
gzip -d -c ../../3rdparty/pip-1.5.6.tar.gz | tar xf -
gzip -d -c ../../3rdparty/wheel-0.24.0.tar.gz | tar xf -
test ! -e ../../3rdparty/wheel-0.24.0.patch || patch -d wheel-0.24.0 -p1 <../../3rdparty/wheel-0.24.0.patch
touch wheel-0.24.0-stamp
gzip -d -c ../../3rdparty/leveldb.tar.gz | tar xf -
test ! -e ../../3rdparty/distribute-0.6.26.patch || patch -d distribute-0.6.26 -p1 <../../3rdparty/distribute-0.6.26.patch
touch distribute-0.6.26-stamp
test ! -e ../../3rdparty/leveldb.patch || patch -d leveldb -p1 <../../3rdparty/leveldb.patch
gzip -d -c ../../3rdparty/zookeeper-3.4.5.tar.gz | tar xf -
patching file build_detect_platform
touch leveldb-stamp
cd leveldb && \
          make  CC="gcc" CXX="g++" OPT="-g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC"
test ! -e ../../3rdparty/pip-1.5.6.patch || patch -d pip-1.5.6 -p1 <../../3rdparty/pip-1.5.6.patch
touch pip-1.5.6-stamp
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/leveldb'
g++ -pthread -shared -Wl,-soname -Wl,/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/leveldb/libleveldb.so.1 -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -fPIC db/builder.cc db/c.cc db/dbformat.cc db/db_impl.cc db/db_iter.cc db/filename.cc db/log_reader.cc db/log_writer.cc db/memtable.cc db/repair.cc db/table_cache.cc db/version_edit.cc db/version_set.cc db/write_batch.cc table/block_builder.cc table/block.cc table/filter_block.cc table/format.cc table/iterator.cc table/merger.cc table/table_builder.cc table/table.cc table/two_level_iterator.cc util/arena.cc util/bloom.cc util/cache.cc util/coding.cc util/comparator.cc util/crc32c.cc util/env.cc util/env_posix.cc util/filter_policy.cc util/hash.cc util/histogram.cc util/logging.cc util/options.cc util/status.cc  port/port_posix.cc -o libleveldb.so.1.4
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/builder.cc -o db/builder.o
test ! -e ../../3rdparty/zookeeper-3.4.5.patch || patch -d zookeeper-3.4.5 -p1 <../../3rdparty/zookeeper-3.4.5.patch
patching file src/c/include/recordio.h
patching file src/c/src/recordio.c
patching file src/c/src/zookeeper.c
patching file src/c/tests/ZKMocks.cc
touch zookeeper-3.4.5-stamp
cd zookeeper-3.4.5/src/c && ./configure '--disable-python-dependency-install' '--srcdir=..' '--prefix=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst' 'JAVA_HOME=/home/jenkins/tools/java/jdk1.6.0_20-64' CXXFLAGS='-g1 -O0 -Wno-unused-local-typedefs -std=c++11' --enable-shared=no --with-pic --srcdir=. && \
	  make 
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/c.cc -o db/c.o
checking for doxygen... no
checking for perl... /usr/bin/perl
checking for dot... no
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/dbformat.cc -o db/dbformat.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/db_impl.cc -o db/db_impl.o
yes
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking for cppunit-config... /usr/bin/cppunit-config
checking for Cppunit - version >= 1.10.2... 1.13.1
checking for generated/zookeeper.jute.c... yes
checking for generated/zookeeper.jute.h... yes
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking whether gcc and cc understand -c and -o together... yes
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln -s works... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for a sed that does not truncate output... /bin/sed
checking for egrep... grep -E
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for /usr/bin/ld option to reload object files... -r
checking for BSD-compatible nm... /usr/bin/nm -B
checking how to recognise dependent libraries... pass_all
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/db_iter.cc -o db/db_iter.o
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/filename.cc -o db/filename.o
yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/log_reader.cc -o db/log_reader.o
yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking how to run the C++ preprocessor... g++ -E
checking for g77... no
checking for f77... no
checking for xlf... no
checking for frt... no
checking for pgf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for f90... no
checking for xlf90... no
checking for pgf90... no
checking for epcf90... no
checking for f95... no
checking for fort... no
checking for xlf95... no
checking for ifc... no
checking for efc... no
checking for pgf95... no
checking for lf95... no
checking for gfortran... no
checking whether we are using the GNU Fortran 77 compiler... no
checking whether  accepts -g... no
checking the maximum length of command line arguments... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/log_writer.cc -o db/log_writer.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/memtable.cc -o db/memtable.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/repair.cc -o db/repair.o
32768
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for objdir... .libs
checking for ar... ar
checking for ranlib... ranlib
checking for strip... strip
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC
checking if gcc PIC flag -fPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
configure: creating libtool
appending configuration tag "CXX" to libtool
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/table_cache.cc -o db/table_cache.o
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC
checking if g++ PIC flag -fPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/version_edit.cc -o db/version_edit.o
yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
appending configuration tag "F77" to libtool
checking for pthread_mutex_lock in -lpthread... yes
configure: building with SyncAPI support
checking for ANSI C header files... (cached) yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking netdb.h usability... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/version_set.cc -o db/version_set.o
yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking sys/utsname.h usability... yes
checking sys/utsname.h presence... yes
checking for sys/utsname.h... yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether time.h and sys/time.h may both be included... yes
checking for nfds_t... yes
checking whether to enable ipv6... yes
checking for getcwd... yes
checking for gethostbyname... yes
checking for gethostname... yes
checking for getlogin... yes
checking for getpwuid_r... yes
checking for gettimeofday... yes
checking for getuid... yes
checking for memmove... yes
checking for memset... yes
checking for poll... yes
checking for socket... yes
checking for strchr... yes
checking for strdup... yes
checking for strerror... g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c db/write_batch.cc -o db/write_batch.o
yes
checking for strtol... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/block_builder.cc -o table/block_builder.o
config.status: executing depfiles commands
make[5]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
make  all-am
make[6]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zookeeper.lo -MD -MP -MF ".deps/zookeeper.Tpo" -c -o zookeeper.lo `test -f 'src/zookeeper.c' || echo './'`src/zookeeper.c; \
	then mv -f ".deps/zookeeper.Tpo" ".deps/zookeeper.Plo"; else rm -f ".deps/zookeeper.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zookeeper.lo -MD -MP -MF .deps/zookeeper.Tpo -c src/zookeeper.c  -fPIC -DPIC -o zookeeper.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/block.cc -o table/block.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/filter_block.cc -o table/filter_block.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/format.cc -o table/format.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT recordio.lo -MD -MP -MF ".deps/recordio.Tpo" -c -o recordio.lo `test -f 'src/recordio.c' || echo './'`src/recordio.c; \
	then mv -f ".deps/recordio.Tpo" ".deps/recordio.Plo"; else rm -f ".deps/recordio.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT recordio.lo -MD -MP -MF .deps/recordio.Tpo -c src/recordio.c  -fPIC -DPIC -o recordio.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/iterator.cc -o table/iterator.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zookeeper.jute.lo -MD -MP -MF ".deps/zookeeper.jute.Tpo" -c -o zookeeper.jute.lo `test -f 'generated/zookeeper.jute.c' || echo './'`generated/zookeeper.jute.c; \
	then mv -f ".deps/zookeeper.jute.Tpo" ".deps/zookeeper.jute.Plo"; else rm -f ".deps/zookeeper.jute.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zookeeper.jute.lo -MD -MP -MF .deps/zookeeper.jute.Tpo -c generated/zookeeper.jute.c  -fPIC -DPIC -o zookeeper.jute.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zk_log.lo -MD -MP -MF ".deps/zk_log.Tpo" -c -o zk_log.lo `test -f 'src/zk_log.c' || echo './'`src/zk_log.c; \
	then mv -f ".deps/zk_log.Tpo" ".deps/zk_log.Plo"; else rm -f ".deps/zk_log.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zk_log.lo -MD -MP -MF .deps/zk_log.Tpo -c src/zk_log.c  -fPIC -DPIC -o zk_log.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT zk_hashtable.lo -MD -MP -MF ".deps/zk_hashtable.Tpo" -c -o zk_hashtable.lo `test -f 'src/zk_hashtable.c' || echo './'`src/zk_hashtable.c; \
	then mv -f ".deps/zk_hashtable.Tpo" ".deps/zk_hashtable.Plo"; else rm -f ".deps/zk_hashtable.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zk_hashtable.lo -MD -MP -MF .deps/zk_hashtable.Tpo -c src/zk_hashtable.c  -fPIC -DPIC -o zk_hashtable.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT st_adaptor.lo -MD -MP -MF ".deps/st_adaptor.Tpo" -c -o st_adaptor.lo `test -f 'src/st_adaptor.c' || echo './'`src/st_adaptor.c; \
	then mv -f ".deps/st_adaptor.Tpo" ".deps/st_adaptor.Plo"; else rm -f ".deps/st_adaptor.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT st_adaptor.lo -MD -MP -MF .deps/st_adaptor.Tpo -c src/st_adaptor.c  -fPIC -DPIC -o st_adaptor.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT hashtable_itr.lo -MD -MP -MF ".deps/hashtable_itr.Tpo" -c -o hashtable_itr.lo `test -f 'src/hashtable/hashtable_itr.c' || echo './'`src/hashtable/hashtable_itr.c; \
	then mv -f ".deps/hashtable_itr.Tpo" ".deps/hashtable_itr.Plo"; else rm -f ".deps/hashtable_itr.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT hashtable_itr.lo -MD -MP -MF .deps/hashtable_itr.Tpo -c src/hashtable/hashtable_itr.c  -fPIC -DPIC -o hashtable_itr.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT hashtable.lo -MD -MP -MF ".deps/hashtable.Tpo" -c -o hashtable.lo `test -f 'src/hashtable/hashtable.c' || echo './'`src/hashtable/hashtable.c; \
	then mv -f ".deps/hashtable.Tpo" ".deps/hashtable.Plo"; else rm -f ".deps/hashtable.Tpo"; exit 1; fi
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.lo -MD -MP -MF ".deps/libzkmt_la-zookeeper.Tpo" -c -o libzkmt_la-zookeeper.lo `test -f 'src/zookeeper.c' || echo './'`src/zookeeper.c; \
	then mv -f ".deps/libzkmt_la-zookeeper.Tpo" ".deps/libzkmt_la-zookeeper.Plo"; else rm -f ".deps/libzkmt_la-zookeeper.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT hashtable.lo -MD -MP -MF .deps/hashtable.Tpo -c src/hashtable/hashtable.c  -fPIC -DPIC -o hashtable.o
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.lo -MD -MP -MF .deps/libzkmt_la-zookeeper.Tpo -c src/zookeeper.c  -fPIC -DPIC -o libzkmt_la-zookeeper.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-recordio.lo -MD -MP -MF ".deps/libzkmt_la-recordio.Tpo" -c -o libzkmt_la-recordio.lo `test -f 'src/recordio.c' || echo './'`src/recordio.c; \
	then mv -f ".deps/libzkmt_la-recordio.Tpo" ".deps/libzkmt_la-recordio.Plo"; else rm -f ".deps/libzkmt_la-recordio.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-recordio.lo -MD -MP -MF .deps/libzkmt_la-recordio.Tpo -c src/recordio.c  -fPIC -DPIC -o libzkmt_la-recordio.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.jute.lo -MD -MP -MF ".deps/libzkmt_la-zookeeper.jute.Tpo" -c -o libzkmt_la-zookeeper.jute.lo `test -f 'generated/zookeeper.jute.c' || echo './'`generated/zookeeper.jute.c; \
	then mv -f ".deps/libzkmt_la-zookeeper.jute.Tpo" ".deps/libzkmt_la-zookeeper.jute.Plo"; else rm -f ".deps/libzkmt_la-zookeeper.jute.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zookeeper.jute.lo -MD -MP -MF .deps/libzkmt_la-zookeeper.jute.Tpo -c generated/zookeeper.jute.c  -fPIC -DPIC -o libzkmt_la-zookeeper.jute.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_log.lo -MD -MP -MF ".deps/libzkmt_la-zk_log.Tpo" -c -o libzkmt_la-zk_log.lo `test -f 'src/zk_log.c' || echo './'`src/zk_log.c; \
	then mv -f ".deps/libzkmt_la-zk_log.Tpo" ".deps/libzkmt_la-zk_log.Plo"; else rm -f ".deps/libzkmt_la-zk_log.Tpo"; exit 1; fi
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_hashtable.lo -MD -MP -MF ".deps/libzkmt_la-zk_hashtable.Tpo" -c -o libzkmt_la-zk_hashtable.lo `test -f 'src/zk_hashtable.c' || echo './'`src/zk_hashtable.c; \
	then mv -f ".deps/libzkmt_la-zk_hashtable.Tpo" ".deps/libzkmt_la-zk_hashtable.Plo"; else rm -f ".deps/libzkmt_la-zk_hashtable.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_log.lo -MD -MP -MF .deps/libzkmt_la-zk_log.Tpo -c src/zk_log.c  -fPIC -DPIC -o libzkmt_la-zk_log.o
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-zk_hashtable.lo -MD -MP -MF .deps/libzkmt_la-zk_hashtable.Tpo -c src/zk_hashtable.c  -fPIC -DPIC -o libzkmt_la-zk_hashtable.o
if /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-mt_adaptor.lo -MD -MP -MF ".deps/libzkmt_la-mt_adaptor.Tpo" -c -o libzkmt_la-mt_adaptor.lo `test -f 'src/mt_adaptor.c' || echo './'`src/mt_adaptor.c; \
	then mv -f ".deps/libzkmt_la-mt_adaptor.Tpo" ".deps/libzkmt_la-mt_adaptor.Plo"; else rm -f ".deps/libzkmt_la-mt_adaptor.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include -I./tests -I./generated -DTHREADED -g -O2 -D_GNU_SOURCE -MT libzkmt_la-mt_adaptor.lo -MD -MP -MF .deps/libzkmt_la-mt_adaptor.Tpo -c src/mt_adaptor.c  -fPIC -DPIC -o libzkmt_la-mt_adaptor.o
if gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -Wall -Werror  -g -O2 -D_GNU_SOURCE -MT cli.o -MD -MP -MF ".deps/cli.Tpo" -c -o cli.o `test -f 'src/cli.c' || echo './'`src/cli.c; \
	then mv -f ".deps/cli.Tpo" ".deps/cli.Po"; else rm -f ".deps/cli.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT cli_mt-cli.o -MD -MP -MF ".deps/cli_mt-cli.Tpo" -c -o cli_mt-cli.o `test -f 'src/cli.c' || echo './'`src/cli.c; \
	then mv -f ".deps/cli_mt-cli.Tpo" ".deps/cli_mt-cli.Po"; else rm -f ".deps/cli_mt-cli.Tpo"; exit 1; fi
if gcc -DHAVE_CONFIG_H -I. -I. -I.  -I./include -I./tests -I./generated  -DTHREADED -g -O2 -D_GNU_SOURCE -MT load_gen-load_gen.o -MD -MP -MF ".deps/load_gen-load_gen.Tpo" -c -o load_gen-load_gen.o `test -f 'src/load_gen.c' || echo './'`src/load_gen.c; \
	then mv -f ".deps/load_gen-load_gen.Tpo" ".deps/load_gen-load_gen.Po"; else rm -f ".deps/load_gen-load_gen.Tpo"; exit 1; fi
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzkst.la   zookeeper.lo recordio.lo zookeeper.jute.lo zk_log.lo zk_hashtable.lo st_adaptor.lo -lm 
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libhashtable.la   hashtable_itr.lo hashtable.lo  
mkdir .libs
ar cru .libs/libzkst.a  zookeeper.o recordio.o zookeeper.jute.o zk_log.o zk_hashtable.o st_adaptor.o
ranlib .libs/libzkst.a
creating libzkst.la
ar cru .libs/libhashtable.a  hashtable_itr.o hashtable.o
ranlib .libs/libhashtable.a
creating libhashtable.la
(cd .libs && rm -f libzkst.la && ln -s ../libzkst.la libzkst.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzkmt.la   libzkmt_la-zookeeper.lo libzkmt_la-recordio.lo libzkmt_la-zookeeper.jute.lo libzkmt_la-zk_log.lo libzkmt_la-zk_hashtable.lo libzkmt_la-mt_adaptor.lo -lm 
(cd .libs && rm -f libhashtable.la && ln -s ../libhashtable.la libhashtable.la)
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/merger.cc -o table/merger.o
ar cru .libs/libzkmt.a  libzkmt_la-zookeeper.o libzkmt_la-recordio.o libzkmt_la-zookeeper.jute.o libzkmt_la-zk_log.o libzkmt_la-zk_hashtable.o libzkmt_la-mt_adaptor.o
ranlib .libs/libzkmt.a
creating libzkmt.la
(cd .libs && rm -f libzkmt.la && ln -s ../libzkmt.la libzkmt.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzookeeper_st.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib -no-undefined -version-info 2 -export-symbols-regex '(zoo_|zookeeper_|zhandle|Z|format_log_message|log_message|logLevel|deallocate_|zerror|is_unrecoverable)'  libzkst.la libhashtable.la 
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/table_builder.cc -o table/table_builder.o
rm -fr .libs/libzookeeper_st.lax
mkdir .libs/libzookeeper_st.lax
rm -fr .libs/libzookeeper_st.lax/libzkst.a
mkdir .libs/libzookeeper_st.lax/libzkst.a
(cd .libs/libzookeeper_st.lax/libzkst.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libzkst.a)
rm -fr .libs/libzookeeper_st.lax/libhashtable.a
mkdir .libs/libzookeeper_st.lax/libhashtable.a
(cd .libs/libzookeeper_st.lax/libhashtable.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libhashtable.a)
ar cru .libs/libzookeeper_st.a   .libs/libzookeeper_st.lax/libzkst.a/zookeeper.o .libs/libzookeeper_st.lax/libzkst.a/zk_hashtable.o .libs/libzookeeper_st.lax/libzkst.a/st_adaptor.o .libs/libzookeeper_st.lax/libzkst.a/zookeeper.jute.o .libs/libzookeeper_st.lax/libzkst.a/recordio.o .libs/libzookeeper_st.lax/libzkst.a/zk_log.o  .libs/libzookeeper_st.lax/libhashtable.a/hashtable_itr.o .libs/libzookeeper_st.lax/libhashtable.a/hashtable.o 
ranlib .libs/libzookeeper_st.a
rm -fr .libs/libzookeeper_st.lax
creating libzookeeper_st.la
(cd .libs && rm -f libzookeeper_st.la && ln -s ../libzookeeper_st.la libzookeeper_st.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o libzookeeper_mt.la -rpath /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib -no-undefined -version-info 2 -export-symbols-regex '(zoo_|zookeeper_|zhandle|Z|format_log_message|log_message|logLevel|deallocate_|zerror|is_unrecoverable)'  libzkmt.la libhashtable.la -lpthread 
rm -fr .libs/libzookeeper_mt.lax
mkdir .libs/libzookeeper_mt.lax
rm -fr .libs/libzookeeper_mt.lax/libzkmt.a
mkdir .libs/libzookeeper_mt.lax/libzkmt.a
(cd .libs/libzookeeper_mt.lax/libzkmt.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libzkmt.a)
rm -fr .libs/libzookeeper_mt.lax/libhashtable.a
mkdir .libs/libzookeeper_mt.lax/libhashtable.a
(cd .libs/libzookeeper_mt.lax/libhashtable.a && ar x /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c/./.libs/libhashtable.a)
ar cru .libs/libzookeeper_mt.a   .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zookeeper.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zk_hashtable.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-mt_adaptor.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zookeeper.jute.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-recordio.o .libs/libzookeeper_mt.lax/libzkmt.a/libzkmt_la-zk_log.o  .libs/libzookeeper_mt.lax/libhashtable.a/hashtable_itr.o .libs/libzookeeper_mt.lax/libhashtable.a/hashtable.o 
ranlib .libs/libzookeeper_mt.a
rm -fr .libs/libzookeeper_mt.lax
creating libzookeeper_mt.la
(cd .libs && rm -f libzookeeper_mt.la && ln -s ../libzookeeper_mt.la libzookeeper_mt.la)
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o cli_st  cli.o libzookeeper_st.la 
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o cli_mt  cli_mt-cli.o libzookeeper_mt.la 
gcc -Wall -Werror -g -O2 -D_GNU_SOURCE -o cli_st cli.o  ./.libs/libzookeeper_st.a -lm  
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/table.cc -o table/table.o
gcc -Wall -Werror -g -O2 -D_GNU_SOURCE -o cli_mt cli_mt-cli.o  ./.libs/libzookeeper_mt.a -lm -lpthread  
/bin/bash ./libtool --tag=CC --mode=link gcc -Wall -Werror  -g -O2 -D_GNU_SOURCE   -o load_gen  load_gen-load_gen.o libzookeeper_mt.la 
gcc -Wall -Werror -g -O2 -D_GNU_SOURCE -o load_gen load_gen-load_gen.o  ./.libs/libzookeeper_mt.a -lm -lpthread  
make[6]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/zookeeper-3.4.5/src/c'
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c table/two_level_iterator.cc -o table/two_level_iterator.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/arena.cc -o util/arena.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/bloom.cc -o util/bloom.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/cache.cc -o util/cache.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/coding.cc -o util/coding.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/comparator.cc -o util/comparator.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/crc32c.cc -o util/crc32c.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/env.cc -o util/env.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/env_posix.cc -o util/env_posix.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/filter_policy.cc -o util/filter_policy.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/hash.cc -o util/hash.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/histogram.cc -o util/histogram.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/logging.cc -o util/logging.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/options.cc -o util/options.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c util/status.cc -o util/status.o
g++ -I. -I./include -fno-builtin-memcmp -pthread -DOS_LINUX -DLEVELDB_PLATFORM_POSIX -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -fPIC -c port/port_posix.cc -o port/port_posix.o
ln -fs libleveldb.so.1.4 libleveldb.so
ln -fs libleveldb.so.1.4 libleveldb.so.1
rm -f libleveldb.a
ar -rs libleveldb.a db/builder.o db/c.o db/dbformat.o db/db_impl.o db/db_iter.o db/filename.o db/log_reader.o db/log_writer.o db/memtable.o db/repair.o db/table_cache.o db/version_edit.o db/version_set.o db/write_batch.o table/block_builder.o table/block.o table/filter_block.o table/format.o table/iterator.o table/merger.o table/table_builder.o table/table.o table/two_level_iterator.o util/arena.o util/bloom.o util/cache.o util/coding.o util/comparator.o util/crc32c.o util/env.o util/env_posix.o util/filter_policy.o util/hash.o util/histogram.o util/logging.o util/options.o util/status.o port/port_posix.o
ar: creating libleveldb.a
make[5]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/leveldb'
make[4]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
make[3]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty'
Making all in src
make[2]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
/bin/mkdir -p .
/bin/mkdir -p containerizer
/bin/mkdir -p fetcher
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/mesos.proto
/bin/mkdir -p ../include/mesos/containerizer
/bin/mkdir -p ../include/mesos/fetcher
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/fetcher/fetcher.proto
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/containerizer/containerizer.proto
mv ../include/mesos/fetcher/*.pb.cc fetcher
mv ../include/mesos/containerizer/*.pb.cc containerizer
/bin/mkdir -p scheduler
/bin/mkdir -p  java/generated/org/apache/mesos
/bin/mkdir -p ../include/mesos/scheduler
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --java_out=java/generated ../../include/mesos/mesos.proto
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=../include ../../include/mesos/scheduler/scheduler.proto
mv ../include/mesos/scheduler/*.pb.cc scheduler
/bin/mkdir -p  java/generated/org/apache/mesos/containerizer
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --java_out=java/generated ../../include/mesos/containerizer/containerizer.proto
mv ../include/mesos/*.pb.cc .
/bin/mkdir -p python/interface/src/mesos/interface
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include/mesos				\
		-I../../include -I../../src --python_out=python/interface/src/mesos/interface ../../include/mesos/mesos.proto
/bin/mkdir -p python/interface/src/mesos/interface
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include/mesos/containerizer		\
		-I../../include -I../../src						\
		--python_out=python/interface/src/mesos/interface ../../include/mesos/containerizer/containerizer.proto
/bin/mkdir -p messages
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/messages/messages.proto
sed -e 's/mesos\.mesos_pb2/mesos_pb2/' <python/interface/src/mesos/interface/containerizer_pb2.py >python/interface/src/mesos/interface/containerizer_pb2.py
/bin/mkdir -p messages
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/messages/log.proto
/bin/mkdir -p messages
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/messages/state.proto
/bin/mkdir -p master
../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/protoc -I../../include -I../../src --cpp_out=. ../../src/master/registry.proto
Building mesos-0.22.0.jar ...
[INFO] Scanning for projects...
[INFO] 
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building mesos 0.22.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ mesos ---
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (default) @ mesos ---
[INFO] Source directory: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/generated added.
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ mesos ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ mesos ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/classes
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ mesos ---
[INFO] Compiling 17 source files to /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ mesos ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/src/test/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ mesos ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ mesos ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ mesos ---
[INFO] Building jar: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/mesos-0.22.0.jar
[INFO] 
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ mesos ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (build-and-attach-javadocs) @ mesos ---
[INFO] 
Loading source files for package org.apache.mesos...
Loading source files for package org.apache.mesos.state...
Loading source files for package org.apache.mesos.containerizer...
Constructing Javadoc information...
Standard Doclet version 1.6.0_20
Building tree for all the packages and classes...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Executor.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//ExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Entry.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.OperationFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Position.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Reader.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.Writer.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Log.WriterFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//MesosExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//MesosSchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//Scheduler.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//SchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//AbstractState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//InMemoryState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//LevelDBState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//LogState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//State.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//Variable.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//ZooKeeperState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/overview-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-summary.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-tree.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-summary.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-tree.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/constant-values.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/serialized-form.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Scheduler.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/SchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/ExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Executor.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Position.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Entry.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.OperationFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.WriterFailedException.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Reader.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/Log.Writer.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/MesosExecutorDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//class-use/MesosSchedulerDriver.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/Variable.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/LogState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/ZooKeeperState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/AbstractState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/InMemoryState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/State.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//class-use/LevelDBState.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos//package-use.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/org/apache/mesos/state//package-use.html...
Building index for all the packages and classes...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/overview-tree.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/index-all.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/deprecated-list.html...
Building index for all classes...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/allclasses-frame.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/allclasses-noframe.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/index.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/overview-summary.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/help-doc.html...
Generating /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/apidocs/stylesheet.css...
3 warnings
[WARNING] Javadoc Warnings
[WARNING] javadoc: warning - No source files for package org.apache.mesos.containerizer
[WARNING] javadoc: warning - No source files for package org.apache.mesos.containerizer
[WARNING] /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/../src/java/src/org/apache/mesos/SchedulerDriver.java:161: warning - Tag @link: can't find launchTasks(Collection, Collection, Filters) in org.apache.mesos.SchedulerDriver
[INFO] Building jar: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/mesos-0.22.0-javadoc.jar
[INFO] 
[INFO] >>> maven-source-plugin:2.1.2:jar (attach-sources) @ mesos >>>
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (default) @ mesos ---
[INFO] Source directory: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/generated added.
[INFO] 
[INFO] <<< maven-source-plugin:2.1.2:jar (attach-sources) @ mesos <<<
[INFO] 
[INFO] --- maven-source-plugin:2.1.2:jar (attach-sources) @ mesos ---
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/mesos already added, skipping
[INFO] Building jar: /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/mesos-0.22.0-sources.jar
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/mesos already added, skipping
[INFO] 
[INFO] --- maven-shade-plugin:2.2:shade (default) @ mesos ---
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Attaching shaded artifact.
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:copy-dependencies (copy-dependencies) @ mesos ---
[INFO] Copying protobuf-java-2.5.0.jar to /home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.341 s
[INFO] Finished at: 2015-02-08T19:57:01+00:00
[INFO] Final Memory: 57M/1302M
[INFO] ------------------------------------------------------------------------
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.MesosSchedulerDriver
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
          org.apache.mesos.MesosExecutorDriver
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.Log
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.AbstractState
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.LevelDBState
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.LogState
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.Variable
/home/jenkins/tools/java/jdk1.6.0_20-64/bin/javah -d java/jni				\
	-classpath java/target/mesos-0.22.0.jar:/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src/java/target/protobuf-java-2.5.0.jar				\
	  org.apache.mesos.state.ZooKeeperState
make  all-am
make[3]: Entering directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-status_update_manager.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Tpo -c -o slave/libmesos_no_3rdparty_la-status_update_manager.lo `test -f 'slave/status_update_manager.cpp' |
 | echo '../../src/'`slave/status_update_manager.cpp
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-linux_launcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-linux_launcher.lo `test -f 'slave/contain
 erizer/linux_launcher.cpp' || echo '../../src/'`slave/containerizer/linux_launcher.cpp
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libmesos_no_3rdparty_la-mesos.pb.lo -MD -MP -MF .deps/libmesos_no_3rdparty_la-mesos.pb.Tpo -c -o libmesos_no_3rdparty_la-mesos.pb.lo `test -f 'mesos.pb.cc' || echo '../../src/'`mesos.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-status_update_manager.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Tpo -c ../../src/slave/status_update_manager.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-status_update_manager.o
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-linux_launcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Tpo -c ../../src/slave/containerizer/linux_launcher.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-linux_
 launcher.o
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT libmesos_no_3rdparty_la-mesos.pb.lo -MD -MP -MF .deps/libmesos_no_3rdparty_la-mesos.pb.Tpo -c mesos.pb.cc  -fPIC -DPIC -o .libs/libmesos_no_3rdparty_la-mesos.pb.o
mv -f .deps/libmesos_no_3rdparty_la-mesos.pb.Tpo .deps/libmesos_no_3rdparty_la-mesos.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.pb.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Tpo -c -o scheduler/libmesos_no_3rdparty_la-scheduler.pb.lo `test -f 'scheduler/scheduler.pb.cc' || echo '../../src/'`s
 cheduler/scheduler.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.pb.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Tpo -c scheduler/scheduler.pb.cc  -fPIC -DPIC -o scheduler/.libs/libmesos_no_3rdparty_la-scheduler.pb.o
mv -f scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Tpo scheduler/.deps/libmesos_no_3rdparty_la-scheduler.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registry.pb.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registry.pb.Tpo -c -o master/libmesos_no_3rdparty_la-registry.pb.lo `test -f 'master/registry.pb.cc' || echo '../../src/'`master/registry.p
 b.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registry.pb.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registry.pb.Tpo -c master/registry.pb.cc  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-registry.pb.o
mv -f master/.deps/libmesos_no_3rdparty_la-registry.pb.Tpo master/.deps/libmesos_no_3rdparty_la-registry.pb.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src    -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins
 /jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -DBUILD_DATE="\"$(date '+%Y-%m-%d %H:%M:%S')\"" -DBUILD_TIME="\"$(date '+%s')\"" -DBUILD_USER="\"$USER\"" -DBUILD_JAVA_JVM_LIBRARY=\"/home/jenkins/tools/java/jdk1.6.0_20-64/jre/lib/amd64/server/libjvm.so\"  -DBUILD_FLAGS="\"$BUILD_FLAGS\"" -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++
 11 -MT common/libbuild_la-build.lo -MD -MP -MF common/.deps/libbuild_la-build.Tpo -c -o common/libbuild_la-build.lo `test -f 'common/build.cpp' || echo '../../src/'`common/build.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src "-DBUILD_DATE=\"2015-02-08 19:57:15\"" -DBUILD_TIME=\"1423425435\" -DBUILD_USER=\"jenkins\" -DBUILD_JAVA_JVM_LIBRARY=\"/home/jenkins/tools/java/jdk1.6.0_20-64/jre/lib/amd64/server/libjvm.so\" -DBUILD_FLAGS=\"\" -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libbuild_la-build.lo -MD -MP -MF common/.deps/lib
 build_la-build.Tpo -c ../../src/common/build.cpp  -fPIC -DPIC -o common/.libs/libbuild_la-build.o
mv -f common/.deps/libbuild_la-build.Tpo common/.deps/libbuild_la-build.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/liblog_la-log.pb.lo -MD -MP -MF messages/.deps/liblog_la-log.pb.Tpo -c -o messages/liblog_la-log.pb.lo `test -f 'messages/log.pb.cc' || echo '../../src/'`messages/log.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/liblog_la-log.pb.lo -MD -MP -MF messages/.deps/liblog_la-log.pb.Tpo -c messages/log.pb.cc  -fPIC -DPIC -o messages/.libs/liblog_la-log.pb.o
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-linux_launcher.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/libstate_la-state.pb.lo -MD -MP -MF messages/.deps/libstate_la-state.pb.Tpo -c -o messages/libstate_la-state.pb.lo `test -f 'messages/state.pb.cc' || echo '../../src/'`messages/state.pb.cc
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT messages/libstate_la-state.pb.lo -MD -MP -MF messages/.deps/libstate_la-state.pb.Tpo -c messages/state.pb.cc  -fPIC -DPIC -o messages/.libs/libstate_la-state.pb.o
mv -f messages/.deps/liblog_la-log.pb.Tpo messages/.deps/liblog_la-log.pb.Plo
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT local/mesos_local-main.o -MD -MP -MF local/.deps/mesos_local-main.Tpo -c -o local/mesos_local-main.o `test -f 'local/main.cpp' || echo '../../src/'`local/main.cpp
mv -f slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Tpo slave/.deps/libmesos_no_3rdparty_la-status_update_manager.Plo
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT log/mesos_log-main.o -MD -MP -MF log/.deps/mesos_log-main.Tpo -c -o log/mesos_log-main.o `test -f 'log/main.cpp' || echo '../../src/'`log/main.cpp
mv -f messages/.deps/libstate_la-state.pb.Tpo messages/.deps/libstate_la-state.pb.Plo
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/mesos/mesos_containerizer-main.o -MD -MP -MF slave/containerizer/mesos/.deps/mesos_containerizer-main.Tpo -c -o slave/containerizer/mesos/mesos_containerizer-main.o `test -f 'slave/containerizer/mesos/main.cpp' || echo '../../src/'`slave/containerizer/mesos/main.
 cpp
mv -f log/.deps/mesos_log-main.Tpo log/.deps/mesos_log-main.Po
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT usage/mesos_usage-main.o -MD -MP -MF usage/.deps/mesos_usage-main.Tpo -c -o usage/mesos_usage-main.o `test -f 'usage/main.cpp' || echo '../../src/'`usage/main.cpp
mv -f slave/containerizer/mesos/.deps/mesos_containerizer-main.Tpo slave/containerizer/mesos/.deps/mesos_containerizer-main.Po
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/mesos_master-main.o -MD -MP -MF master/.deps/mesos_master-main.Tpo -c -o master/mesos_master-main.o `test -f 'master/main.cpp' || echo '../../src/'`master/main.cpp
mv -f usage/.deps/mesos_usage-main.Tpo usage/.deps/mesos_usage-main.Po
g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.2
 2.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/mesos_slave-main.o -MD -MP -MF slave/.deps/mesos_slave-main.Tpo -c -o slave/mesos_slave-main.o `test -f 'slave/main.cpp' || echo '../../src/'`slave/main.cpp
mv -f local/.deps/mesos_local-main.Tpo local/.deps/mesos_local-main.Po
test "../.." = ".." ||			\
		(/bin/mkdir -p python/src/mesos && cp -pf ../../src/python/src/mesos/__init__.py python/src/mesos/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos && cp -pf ../../src/python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/interface/src/mesos/interface && cp -pf ../../src/python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos && cp -pf ../../src/python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/module.cpp python/native/src/mesos/native/module.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/module.hpp python/native/src/mesos/native/module.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_executor.cpp python/native/src/mesos/native/proxy_executor.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_executor.hpp python/native/src/mesos/native/proxy_executor.hpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.cpp python/native/src/mesos/native/proxy_scheduler.cpp)
test "../.." = ".." ||			\
		(/bin/mkdir -p python/native/src/mesos/native && cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.hpp python/native/src/mesos/native/proxy_scheduler.hpp)
Building protobuf Python egg ...
cd ../3rdparty/libprocess/3rdparty/protobuf-2.5.0/python && \
          CC="gcc" CXX="g++" CFLAGS="-g1 -O0 -Wno-unused-local-typedefs" CXXFLAGS="-g1 -O0 -Wno-unused-local-typedefs -std=c++11" \
          PYTHONPATH=/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/3rdparty/distribute-0.6.26 /usr/bin/python setup.py build bdist_egg
running build
running build_py
Generating google/protobuf/descriptor_pb2.py...
Generating google/protobuf/compiler/plugin_pb2.py...
Generating google/protobuf/unittest_pb2.py...
Generating google/protobuf/unittest_custom_options_pb2.py...
Generating google/protobuf/unittest_import_pb2.py...
Generating google/protobuf/unittest_import_public_pb2.py...
Generating google/protobuf/unittest_mset_pb2.py...
Generating google/protobuf/unittest_no_generic_services_pb2.py...
Generating google/protobuf/internal/test_bad_identifiers_pb2.py...
Generating google/protobuf/internal/more_extensions_pb2.py...
Generating google/protobuf/internal/more_extensions_dynamic_pb2.py...
Generating google/protobuf/internal/more_messages_pb2.py...
Generating google/protobuf/internal/factory_test1_pb2.py...
Generating google/protobuf/internal/factory_test2_pb2.py...
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/google
creating build/lib.linux-x86_64-2.7/google/protobuf
creating build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/__init__.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/api_implementation.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/containers.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/cpp_message.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/decoder.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/encoder.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/enum_type_wrapper.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/message_listener.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/python_message.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/type_checkers.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/internal/wire_format.py -> build/lib.linux-x86_64-2.7/google/protobuf/internal
copying google/protobuf/__init__.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor_pb2.py -> build/lib.linux-x86_64-2.7/google/protobuf
creating build/lib.linux-x86_64-2.7/google/protobuf/compiler
copying google/protobuf/compiler/__init__.py -> build/lib.linux-x86_64-2.7/google/protobuf/compiler
copying google/protobuf/compiler/plugin_pb2.py -> build/lib.linux-x86_64-2.7/google/protobuf/compiler
copying google/protobuf/message.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor_database.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/descriptor_pool.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/message_factory.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/reflection.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/service.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/service_reflection.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/protobuf/text_format.py -> build/lib.linux-x86_64-2.7/google/protobuf
copying google/__init__.py -> build/lib.linux-x86_64-2.7/google
running bdist_egg
running egg_info
creating protobuf.egg-info
writing requirements to protobuf.egg-info/requires.txt
writing protobuf.egg-info/PKG-INFO
writing namespace_packages to protobuf.egg-info/namespace_packages.txt
writing top-level names to protobuf.egg-info/top_level.txt
writing dependency_links to protobuf.egg-info/dependency_links.txt
writing requirements to protobuf.egg-info/requires.txt
writing protobuf.egg-info/PKG-INFO
writing namespace_packages to protobuf.egg-info/namespace_packages.txt
writing top-level names to protobuf.egg-info/top_level.txt
writing dependency_links to protobuf.egg-info/dependency_links.txt
writing manifest file 'protobuf.egg-info/SOURCES.txt'
reading manifest file 'protobuf.egg-info/SOURCES.txt'
writing manifest file 'protobuf.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/google
copying build/lib.linux-x86_64-2.7/google/__init__.py -> build/bdist.linux-x86_64/egg/google
creating build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/service.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/service_reflection.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/message_factory.py -> build/bdist.linux-x86_64/egg/google/protobuf
creating build/bdist.linux-x86_64/egg/google/protobuf/compiler
copying build/lib.linux-x86_64-2.7/google/protobuf/compiler/plugin_pb2.py -> build/bdist.linux-x86_64/egg/google/protobuf/compiler
copying build/lib.linux-x86_64-2.7/google/protobuf/compiler/__init__.py -> build/bdist.linux-x86_64/egg/google/protobuf/compiler
copying build/lib.linux-x86_64-2.7/google/protobuf/reflection.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/__init__.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor_pb2.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/text_format.py -> build/bdist.linux-x86_64/egg/google/protobuf
creating build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/api_implementation.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/enum_type_wrapper.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/type_checkers.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/decoder.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/message_listener.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/cpp_message.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/__init__.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/containers.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/encoder.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/wire_format.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/internal/python_message.py -> build/bdist.linux-x86_64/egg/google/protobuf/internal
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor_database.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/message.py -> build/bdist.linux-x86_64/egg/google/protobuf
copying build/lib.linux-x86_64-2.7/google/protobuf/descriptor_pool.py -> build/bdist.linux-x86_64/egg/google/protobuf
byte-compiling build/bdist.linux-x86_64/egg/google/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/service.py to service.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/service_reflection.py to service_reflection.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/message_factory.py to message_factory.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/compiler/plugin_pb2.py to plugin_pb2.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/compiler/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/reflection.py to reflection.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_pb2.py to descriptor_pb2.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/text_format.py to text_format.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/api_implementation.py to api_implementation.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/enum_type_wrapper.py to enum_type_wrapper.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/type_checkers.py to type_checkers.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/decoder.py to decoder.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/message_listener.py to message_listener.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/cpp_message.py to cpp_message.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/containers.py to containers.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/encoder.py to encoder.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/wire_format.py to wire_format.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/internal/python_message.py to python_message.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor.py to descriptor.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_database.py to descriptor_database.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/message.py to message.pyc
byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_pool.py to descriptor_pool.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/namespace_packages.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying protobuf.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/protobuf-2.5.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authentication/cram_md5/libmesos_no_3rdparty_la-auxprop.lo -MD -MP -MF authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Tpo -c -o authentication/cram_md5/libmesos_no_3rdparty_la-auxprop.lo `test -f 'authentication/cram_md
 5/auxprop.cpp' || echo '../../src/'`authentication/cram_md5/auxprop.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authentication/cram_md5/libmesos_no_3rdparty_la-auxprop.lo -MD -MP -MF authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Tpo -c ../../src/authentication/cram_md5/auxprop.cpp  -fPIC -DPIC -o authentication/cram_md5/.libs/libmesos_no_3rdparty_la-auxprop.o
mv -f authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Tpo authentication/cram_md5/.deps/libmesos_no_3rdparty_la-auxprop.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authorizer/libmesos_no_3rdparty_la-authorizer.lo -MD -MP -MF authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Tpo -c -o authorizer/libmesos_no_3rdparty_la-authorizer.lo `test -f 'authorizer/authorizer.cpp' || echo '../../src/'`auth
 orizer/authorizer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT authorizer/libmesos_no_3rdparty_la-authorizer.lo -MD -MP -MF authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Tpo -c ../../src/authorizer/authorizer.cpp  -fPIC -DPIC -o authorizer/.libs/libmesos_no_3rdparty_la-authorizer.o
mv -f master/.deps/mesos_master-main.Tpo master/.deps/mesos_master-main.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-attributes.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-attributes.Tpo -c -o common/libmesos_no_3rdparty_la-attributes.lo `test -f 'common/attributes.cpp' || echo '../../src/'`common/attributes.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-attributes.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-attributes.Tpo -c ../../src/common/attributes.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-attributes.o
mv -f common/.deps/libmesos_no_3rdparty_la-attributes.Tpo common/.deps/libmesos_no_3rdparty_la-attributes.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-date_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-date_utils.Tpo -c -o common/libmesos_no_3rdparty_la-date_utils.lo `test -f 'common/date_utils.cpp' || echo '../../src/'`common/date_utils.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-date_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-date_utils.Tpo -c ../../src/common/date_utils.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-date_utils.o
mv -f authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Tpo authorizer/.deps/libmesos_no_3rdparty_la-authorizer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-http.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-http.Tpo -c -o common/libmesos_no_3rdparty_la-http.lo `test -f 'common/http.cpp' || echo '../../src/'`common/http.cpp
mv -f common/.deps/libmesos_no_3rdparty_la-date_utils.Tpo common/.deps/libmesos_no_3rdparty_la-date_utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-lock.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-lock.Tpo -c -o common/libmesos_no_3rdparty_la-lock.lo `test -f 'common/lock.cpp' || echo '../../src/'`common/lock.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-http.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-http.Tpo -c ../../src/common/http.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-http.o
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-lock.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-lock.Tpo -c ../../src/common/lock.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-lock.o
mv -f common/.deps/libmesos_no_3rdparty_la-lock.Tpo common/.deps/libmesos_no_3rdparty_la-lock.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-protobuf_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Tpo -c -o common/libmesos_no_3rdparty_la-protobuf_utils.lo `test -f 'common/protobuf_utils.cpp' || echo '../../src/'`comm
 on/protobuf_utils.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-protobuf_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Tpo -c ../../src/common/protobuf_utils.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-protobuf_utils.o
mv -f slave/.deps/mesos_slave-main.Tpo slave/.deps/mesos_slave-main.Po
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-resources.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-resources.Tpo -c -o common/libmesos_no_3rdparty_la-resources.lo `test -f 'common/resources.cpp' || echo '../../src/'`common/resources.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-resources.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-resources.Tpo -c ../../src/common/resources.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-resources.o
mv -f common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Tpo common/.deps/libmesos_no_3rdparty_la-protobuf_utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-thread.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-thread.Tpo -c -o common/libmesos_no_3rdparty_la-thread.lo `test -f 'common/thread.cpp' || echo '../../src/'`common/thread.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-thread.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-thread.Tpo -c ../../src/common/thread.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-thread.o
mv -f common/.deps/libmesos_no_3rdparty_la-thread.Tpo common/.deps/libmesos_no_3rdparty_la-thread.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-type_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-type_utils.Tpo -c -o common/libmesos_no_3rdparty_la-type_utils.lo `test -f 'common/type_utils.cpp' || echo '../../src/'`common/type_utils.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-type_utils.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-type_utils.Tpo -c ../../src/common/type_utils.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-type_utils.o
mv -f common/.deps/libmesos_no_3rdparty_la-http.Tpo common/.deps/libmesos_no_3rdparty_la-http.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-values.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-values.Tpo -c -o common/libmesos_no_3rdparty_la-values.lo `test -f 'common/values.cpp' || echo '../../src/'`common/values.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT common/libmesos_no_3rdparty_la-values.lo -MD -MP -MF common/.deps/libmesos_no_3rdparty_la-values.Tpo -c ../../src/common/values.cpp  -fPIC -DPIC -o common/.libs/libmesos_no_3rdparty_la-values.o
mv -f common/.deps/libmesos_no_3rdparty_la-resources.Tpo common/.deps/libmesos_no_3rdparty_la-resources.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT docker/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF docker/.deps/libmesos_no_3rdparty_la-docker.Tpo -c -o docker/libmesos_no_3rdparty_la-docker.lo `test -f 'docker/docker.cpp' || echo '../../src/'`docker/docker.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT docker/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF docker/.deps/libmesos_no_3rdparty_la-docker.Tpo -c ../../src/docker/docker.cpp  -fPIC -DPIC -o docker/.libs/libmesos_no_3rdparty_la-docker.o
mv -f common/.deps/libmesos_no_3rdparty_la-type_utils.Tpo common/.deps/libmesos_no_3rdparty_la-type_utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT exec/libmesos_no_3rdparty_la-exec.lo -MD -MP -MF exec/.deps/libmesos_no_3rdparty_la-exec.Tpo -c -o exec/libmesos_no_3rdparty_la-exec.lo `test -f 'exec/exec.cpp' || echo '../../src/'`exec/exec.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT exec/libmesos_no_3rdparty_la-exec.lo -MD -MP -MF exec/.deps/libmesos_no_3rdparty_la-exec.Tpo -c ../../src/exec/exec.cpp  -fPIC -DPIC -o exec/.libs/libmesos_no_3rdparty_la-exec.o
mv -f common/.deps/libmesos_no_3rdparty_la-values.Tpo common/.deps/libmesos_no_3rdparty_la-values.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT files/libmesos_no_3rdparty_la-files.lo -MD -MP -MF files/.deps/libmesos_no_3rdparty_la-files.Tpo -c -o files/libmesos_no_3rdparty_la-files.lo `test -f 'files/files.cpp' || echo '../../src/'`files/files.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT files/libmesos_no_3rdparty_la-files.lo -MD -MP -MF files/.deps/libmesos_no_3rdparty_la-files.Tpo -c ../../src/files/files.cpp  -fPIC -DPIC -o files/.libs/libmesos_no_3rdparty_la-files.o
mv -f files/.deps/libmesos_no_3rdparty_la-files.Tpo files/.deps/libmesos_no_3rdparty_la-files.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT hook/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF hook/.deps/libmesos_no_3rdparty_la-manager.Tpo -c -o hook/libmesos_no_3rdparty_la-manager.lo `test -f 'hook/manager.cpp' || echo '../../src/'`hook/manager.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT hook/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF hook/.deps/libmesos_no_3rdparty_la-manager.Tpo -c ../../src/hook/manager.cpp  -fPIC -DPIC -o hook/.libs/libmesos_no_3rdparty_la-manager.o
mv -f exec/.deps/libmesos_no_3rdparty_la-exec.Tpo exec/.deps/libmesos_no_3rdparty_la-exec.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT local/libmesos_no_3rdparty_la-local.lo -MD -MP -MF local/.deps/libmesos_no_3rdparty_la-local.Tpo -c -o local/libmesos_no_3rdparty_la-local.lo `test -f 'local/local.cpp' || echo '../../src/'`local/local.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT local/libmesos_no_3rdparty_la-local.lo -MD -MP -MF local/.deps/libmesos_no_3rdparty_la-local.Tpo -c ../../src/local/local.cpp  -fPIC -DPIC -o local/.libs/libmesos_no_3rdparty_la-local.o
mv -f docker/.deps/libmesos_no_3rdparty_la-docker.Tpo docker/.deps/libmesos_no_3rdparty_la-docker.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging/libmesos_no_3rdparty_la-logging.lo -MD -MP -MF logging/.deps/libmesos_no_3rdparty_la-logging.Tpo -c -o logging/libmesos_no_3rdparty_la-logging.lo `test -f 'logging/logging.cpp' || echo '../../src/'`logging/logging.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT logging/libmesos_no_3rdparty_la-logging.lo -MD -MP -MF logging/.deps/libmesos_no_3rdparty_la-logging.Tpo -c ../../src/logging/logging.cpp  -fPIC -DPIC -o logging/.libs/libmesos_no_3rdparty_la-logging.o
mv -f logging/.deps/libmesos_no_3rdparty_la-logging.Tpo logging/.deps/libmesos_no_3rdparty_la-logging.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-contender.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-contender.Tpo -c -o master/libmesos_no_3rdparty_la-contender.lo `test -f 'master/contender.cpp' || echo '../../src/'`master/contender.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-contender.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-contender.Tpo -c ../../src/master/contender.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-contender.o
mv -f hook/.deps/libmesos_no_3rdparty_la-manager.Tpo hook/.deps/libmesos_no_3rdparty_la-manager.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-constants.Tpo -c -o master/libmesos_no_3rdparty_la-constants.lo `test -f 'master/constants.cpp' || echo '../../src/'`master/constants.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-constants.Tpo -c ../../src/master/constants.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-constants.o
mv -f master/.deps/libmesos_no_3rdparty_la-constants.Tpo master/.deps/libmesos_no_3rdparty_la-constants.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-detector.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-detector.Tpo -c -o master/libmesos_no_3rdparty_la-detector.lo `test -f 'master/detector.cpp' || echo '../../src/'`master/detector.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-detector.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-detector.Tpo -c ../../src/master/detector.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-detector.o
mv -f local/.deps/libmesos_no_3rdparty_la-local.Tpo local/.deps/libmesos_no_3rdparty_la-local.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-drf_sorter.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-drf_sorter.Tpo -c -o master/libmesos_no_3rdparty_la-drf_sorter.lo `test -f 'master/drf_sorter.cpp' || echo '../../src/'`master/drf_sorter.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-drf_sorter.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-drf_sorter.Tpo -c ../../src/master/drf_sorter.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-drf_sorter.o
mv -f master/.deps/libmesos_no_3rdparty_la-contender.Tpo master/.deps/libmesos_no_3rdparty_la-contender.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-http.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-http.Tpo -c -o master/libmesos_no_3rdparty_la-http.lo `test -f 'master/http.cpp' || echo '../../src/'`master/http.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-http.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-http.Tpo -c ../../src/master/http.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-http.o
mv -f master/.deps/libmesos_no_3rdparty_la-drf_sorter.Tpo master/.deps/libmesos_no_3rdparty_la-drf_sorter.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-master.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-master.Tpo -c -o master/libmesos_no_3rdparty_la-master.lo `test -f 'master/master.cpp' || echo '../../src/'`master/master.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-master.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-master.Tpo -c ../../src/master/master.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-master.o
mv -f master/.deps/libmesos_no_3rdparty_la-detector.Tpo master/.deps/libmesos_no_3rdparty_la-detector.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c -o master/libmesos_no_3rdparty_la-metrics.lo `test -f 'master/metrics.cpp' || echo '../../src/'`master/metrics.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c ../../src/master/metrics.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-metrics.o
mv -f master/.deps/libmesos_no_3rdparty_la-http.Tpo master/.deps/libmesos_no_3rdparty_la-http.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registrar.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registrar.Tpo -c -o master/libmesos_no_3rdparty_la-registrar.lo `test -f 'master/registrar.cpp' || echo '../../src/'`master/registrar.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-registrar.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-registrar.Tpo -c ../../src/master/registrar.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-registrar.o
mv -f master/.deps/libmesos_no_3rdparty_la-metrics.Tpo master/.deps/libmesos_no_3rdparty_la-metrics.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-repairer.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-repairer.Tpo -c -o master/libmesos_no_3rdparty_la-repairer.lo `test -f 'master/repairer.cpp' || echo '../../src/'`master/repairer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-repairer.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-repairer.Tpo -c ../../src/master/repairer.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-repairer.o
mv -f master/.deps/libmesos_no_3rdparty_la-repairer.Tpo master/.deps/libmesos_no_3rdparty_la-repairer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-validation.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-validation.Tpo -c -o master/libmesos_no_3rdparty_la-validation.lo `test -f 'master/validation.cpp' || echo '../../src/'`master/validation.cp
 p
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT master/libmesos_no_3rdparty_la-validation.lo -MD -MP -MF master/.deps/libmesos_no_3rdparty_la-validation.Tpo -c ../../src/master/validation.cpp  -fPIC -DPIC -o master/.libs/libmesos_no_3rdparty_la-validation.o
mv -f master/.deps/libmesos_no_3rdparty_la-registrar.Tpo master/.deps/libmesos_no_3rdparty_la-registrar.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT module/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF module/.deps/libmesos_no_3rdparty_la-manager.Tpo -c -o module/libmesos_no_3rdparty_la-manager.lo `test -f 'module/manager.cpp' || echo '../../src/'`module/manager.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT module/libmesos_no_3rdparty_la-manager.lo -MD -MP -MF module/.deps/libmesos_no_3rdparty_la-manager.Tpo -c ../../src/module/manager.cpp  -fPIC -DPIC -o module/.libs/libmesos_no_3rdparty_la-manager.o
mv -f master/.deps/libmesos_no_3rdparty_la-validation.Tpo master/.deps/libmesos_no_3rdparty_la-validation.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-constants.Tpo -c -o sched/libmesos_no_3rdparty_la-constants.lo `test -f 'sched/constants.cpp' || echo '../../src/'`sched/constants.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-constants.Tpo -c ../../src/sched/constants.cpp  -fPIC -DPIC -o sched/.libs/libmesos_no_3rdparty_la-constants.o
mv -f sched/.deps/libmesos_no_3rdparty_la-constants.Tpo sched/.deps/libmesos_no_3rdparty_la-constants.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-sched.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-sched.Tpo -c -o sched/libmesos_no_3rdparty_la-sched.lo `test -f 'sched/sched.cpp' || echo '../../src/'`sched/sched.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT sched/libmesos_no_3rdparty_la-sched.lo -MD -MP -MF sched/.deps/libmesos_no_3rdparty_la-sched.Tpo -c ../../src/sched/sched.cpp  -fPIC -DPIC -o sched/.libs/libmesos_no_3rdparty_la-sched.o
mv -f module/.deps/libmesos_no_3rdparty_la-manager.Tpo module/.deps/libmesos_no_3rdparty_la-manager.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Tpo -c -o scheduler/libmesos_no_3rdparty_la-scheduler.lo `test -f 'scheduler/scheduler.cpp' || echo '../../src/'`scheduler/sc
 heduler.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT scheduler/libmesos_no_3rdparty_la-scheduler.lo -MD -MP -MF scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Tpo -c ../../src/scheduler/scheduler.cpp  -fPIC -DPIC -o scheduler/.libs/libmesos_no_3rdparty_la-scheduler.o
mv -f master/.deps/libmesos_no_3rdparty_la-master.Tpo master/.deps/libmesos_no_3rdparty_la-master.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-constants.Tpo -c -o slave/libmesos_no_3rdparty_la-constants.lo `test -f 'slave/constants.cpp' || echo '../../src/'`slave/constants.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-constants.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-constants.Tpo -c ../../src/slave/constants.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-constants.o
mv -f slave/.deps/libmesos_no_3rdparty_la-constants.Tpo slave/.deps/libmesos_no_3rdparty_la-constants.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-gc.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-gc.Tpo -c -o slave/libmesos_no_3rdparty_la-gc.lo `test -f 'slave/gc.cpp' || echo '../../src/'`slave/gc.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-gc.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-gc.Tpo -c ../../src/slave/gc.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-gc.o
mv -f slave/.deps/libmesos_no_3rdparty_la-gc.Tpo slave/.deps/libmesos_no_3rdparty_la-gc.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-graceful_shutdown.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Tpo -c -o slave/libmesos_no_3rdparty_la-graceful_shutdown.lo `test -f 'slave/graceful_shutdown.cpp' || echo '../../sr
 c/'`slave/graceful_shutdown.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-graceful_shutdown.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Tpo -c ../../src/slave/graceful_shutdown.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-graceful_shutdown.o
mv -f scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Tpo scheduler/.deps/libmesos_no_3rdparty_la-scheduler.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-http.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-http.Tpo -c -o slave/libmesos_no_3rdparty_la-http.lo `test -f 'slave/http.cpp' || echo '../../src/'`slave/http.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-http.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-http.Tpo -c ../../src/slave/http.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-http.o
mv -f sched/.deps/libmesos_no_3rdparty_la-sched.Tpo sched/.deps/libmesos_no_3rdparty_la-sched.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c -o slave/libmesos_no_3rdparty_la-metrics.lo `test -f 'slave/metrics.cpp' || echo '../../src/'`slave/metrics.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-metrics.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-metrics.Tpo -c ../../src/slave/metrics.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-metrics.o
mv -f slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Tpo slave/.deps/libmesos_no_3rdparty_la-graceful_shutdown.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-monitor.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-monitor.Tpo -c -o slave/libmesos_no_3rdparty_la-monitor.lo `test -f 'slave/monitor.cpp' || echo '../../src/'`slave/monitor.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-monitor.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-monitor.Tpo -c ../../src/slave/monitor.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-monitor.o
mv -f slave/.deps/libmesos_no_3rdparty_la-http.Tpo slave/.deps/libmesos_no_3rdparty_la-http.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-paths.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-paths.Tpo -c -o slave/libmesos_no_3rdparty_la-paths.lo `test -f 'slave/paths.cpp' || echo '../../src/'`slave/paths.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-paths.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-paths.Tpo -c ../../src/slave/paths.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-paths.o
mv -f slave/.deps/libmesos_no_3rdparty_la-metrics.Tpo slave/.deps/libmesos_no_3rdparty_la-metrics.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-state.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-state.Tpo -c -o slave/libmesos_no_3rdparty_la-state.lo `test -f 'slave/state.cpp' || echo '../../src/'`slave/state.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-state.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-state.Tpo -c ../../src/slave/state.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-state.o
mv -f slave/.deps/libmesos_no_3rdparty_la-paths.Tpo slave/.deps/libmesos_no_3rdparty_la-paths.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-slave.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-slave.Tpo -c -o slave/libmesos_no_3rdparty_la-slave.lo `test -f 'slave/slave.cpp' || echo '../../src/'`slave/slave.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/libmesos_no_3rdparty_la-slave.lo -MD -MP -MF slave/.deps/libmesos_no_3rdparty_la-slave.Tpo -c ../../src/slave/slave.cpp  -fPIC -DPIC -o slave/.libs/libmesos_no_3rdparty_la-slave.o
mv -f slave/.deps/libmesos_no_3rdparty_la-monitor.Tpo slave/.deps/libmesos_no_3rdparty_la-monitor.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-containerizer.lo `test -f 'slave/containeri
 zer/containerizer.cpp' || echo '../../src/'`slave/containerizer/containerizer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Tpo -c ../../src/slave/containerizer/containerizer.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-container
 izer.o
mv -f slave/.deps/libmesos_no_3rdparty_la-state.Tpo slave/.deps/libmesos_no_3rdparty_la-state.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-composing.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-composing.lo `test -f 'slave/containerizer/composin
 g.cpp' || echo '../../src/'`slave/containerizer/composing.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-composing.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Tpo -c ../../src/slave/containerizer/composing.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-composing.o
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-containerizer.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-docker.lo `test -f 'slave/containerizer/docker.cpp' || ec
 ho '../../src/'`slave/containerizer/docker.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-docker.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Tpo -c ../../src/slave/containerizer/docker.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-docker.o
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-composing.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-external_containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-external_containerizer.lo
  `test -f 'slave/containerizer/external_containerizer.cpp' || echo '../../src/'`slave/containerizer/external_containerizer.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-external_containerizer.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Tpo -c ../../src/slave/containerizer/external_containerizer.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmes
 os_no_3rdparty_la-external_containerizer.o
mv -f slave/.deps/libmesos_no_3rdparty_la-slave.Tpo slave/.deps/libmesos_no_3rdparty_la-slave.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-fetcher.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo `test -f 'slave/containerizer/fetcher.cpp' |
 | echo '../../src/'`slave/containerizer/fetcher.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-fetcher.Tpo -c ../../src/slave/containerizer/fetcher.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-fetcher.o
../../src/slave/containerizer/fetcher.cpp: In member function 'process::Future<Nothing> mesos::slave::FetcherProcess::reserveCacheSpace(const Bytes&, const string&)':
../../src/slave/containerizer/fetcher.cpp:457:34: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
   for (int i = 0; i < cache.size(); i++) {
                                  ^
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-docker.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" -DPACKAGE_STRING=\"mesos\ 0.22.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/
 jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated  -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-isolator.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Tpo -c -o slave/containerizer/libmesos_no_3rdparty_la-isolator.lo `test -f 'slave/containerizer/isolator.cp
 p' || echo '../../src/'`slave/containerizer/isolator.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"0.22.0\" "-DPACKAGE_STRING=\"mesos 0.22.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"0.22.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_LIBCURL=1 -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBSASL2=1 -DMESOS_HAS_JAVA=1 -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src -Wall -Werror -DLIBDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/lib\" -DPKGLIBEXECDIR=\"/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_inst/libexec/mesos\" -DPKGDATADIR=\"/home/jenkins/jenkins-slave/workspace/mesos-r
 eviewbot/mesos-0.22.0/_inst/share/mesos\" -I../../include -I../../3rdparty/libprocess/include -I../../3rdparty/libprocess/3rdparty/stout/include -I../include -I../include/mesos -I../3rdparty/libprocess/3rdparty/boost-1.53.0 -I../3rdparty/libprocess/3rdparty/picojson-4f93734 -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/libprocess/3rdparty/glog-0.3.3/src -I../3rdparty/leveldb/include -I../3rdparty/zookeeper-3.4.5/src/c/include -I../3rdparty/zookeeper-3.4.5/src/c/generated -I../3rdparty/libprocess/3rdparty/protobuf-2.5.0/src -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT slave/containerizer/libmesos_no_3rdparty_la-isolator.lo -MD -MP -MF slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Tpo -c ../../src/slave/containerizer/isolator.cpp  -fPIC -DPIC -o slave/containerizer/.libs/libmesos_no_3rdparty_la-isolator.o
cc1plus: all warnings being treated as errors
make[3]: *** [slave/containerizer/libmesos_no_3rdparty_la-fetcher.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-external_containerizer.Plo
mv -f slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Tpo slave/containerizer/.deps/libmesos_no_3rdparty_la-isolator.Plo
make[3]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/mesos-reviewbot/mesos-0.22.0/_build'
make: *** [distcheck] Error 1

- Mesos ReviewBot


On Feb. 8, 2015, 7:34 p.m., Bernd Mathiske wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30774/
> -----------------------------------------------------------
> 
> (Updated Feb. 8, 2015, 7:34 p.m.)
> 
> 
> Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.
> 
> 
> Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
>     https://issues.apache.org/jira/browse/MESOS-2057
>     https://issues.apache.org/jira/browse/MESOS-2069
>     https://issues.apache.org/jira/browse/MESOS-2070
>     https://issues.apache.org/jira/browse/MESOS-2072
>     https://issues.apache.org/jira/browse/MESOS-2074
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.
> 
> 
> Diffs
> -----
> 
>   include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
>   include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
>   src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
>   src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
>   src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
>   src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
>   src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
>   src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
>   src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
>   src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
>   src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
>   src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
>   src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
>   src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
>   src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
>   src/tests/fetcher_cache_tests.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
>   src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
>   src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
>   src/tests/mock_hadoop.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30774/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>


Re: Review Request 30774: Fetcher Cache

Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30774/
-----------------------------------------------------------

(Updated Feb. 8, 2015, 11:34 a.m.)


Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen.


Changes
-------

Changed title. Now an official request relating to JIRAs.


Summary (updated)
-----------------

Fetcher Cache


Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, and MESOS-2074
    https://issues.apache.org/jira/browse/MESOS-2057
    https://issues.apache.org/jira/browse/MESOS-2069
    https://issues.apache.org/jira/browse/MESOS-2070
    https://issues.apache.org/jira/browse/MESOS-2072
    https://issues.apache.org/jira/browse/MESOS-2074


Repository: mesos


Description (updated)
-------

Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those.


Diffs
-----

  include/mesos/fetcher/fetcher.proto facb87b92bf3194516f636dcc348e136af537721 
  include/mesos/mesos.proto 3a2921dff856d37455593dcbf7340aa537997d35 
  src/Makefile.am 0fe4809542a7d23785a221901947771320b43d52 
  src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b 
  src/launcher/fetcher.cpp 5b2d86d1867b25bf71461fd73df91b089002325b 
  src/slave/constants.hpp 7717abd4f9a3bd6fdca6af2364864e374ce2e056 
  src/slave/containerizer/docker.hpp 70114dcef7a416ae904e95eb9ce365bcd866aebc 
  src/slave/containerizer/docker.cpp 813ddf8c98622748b2da1b739bf122387abc4c79 
  src/slave/containerizer/fetcher.hpp bfd98dbe16e2bd5df3e2c8e9b10e303654f33446 
  src/slave/containerizer/fetcher.cpp 6e6bce08d76bb8a5813c905e3ffeff9b2411fd6d 
  src/slave/containerizer/mesos/containerizer.hpp b3aafe4efa9f3469d7a3fd39243ad66b46d6a54d 
  src/slave/containerizer/mesos/containerizer.cpp fa40d47aee7803833bcde6cce1e86a21d7cf27d0 
  src/slave/flags.hpp f6033355d129f0013d39dd053455c936596bf159 
  src/slave/slave.cpp fff2d725fe49eee984d9151cfb2131202c47994f 
  src/tests/docker_containerizer_tests.cpp 8b212d4e6ed5a179ebadce1bdbbf3edde87d706c 
  src/tests/fetcher_cache_tests.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp fcbf7ad912f0b1b3d106a75a9dcb8213a0c69c07 
  src/tests/mesos.hpp 83a369968ab2403fa341829ac5d11f7243095190 
  src/tests/mesos.cpp 21a405366f56c963611324076efe775f85b9d9f7 
  src/tests/mock_hadoop.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/30774/diff/


Testing (updated)
-------

make check


Thanks,

Bernd Mathiske