You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by ji...@apache.org on 2017/08/10 23:55:30 UTC
[01/11] mesos git commit: Updated LICENSE information for grpc 1.4.2.
Repository: mesos
Updated Branches:
refs/heads/master b9e27f236 -> e08593ae9
Updated LICENSE information for grpc 1.4.2.
Content copied from:
https://github.com/grpc/grpc/blob/master/LICENSE.
Review: https://reviews.apache.org/r/61095
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/b082dd80
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/b082dd80
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/b082dd80
Branch: refs/heads/master
Commit: b082dd80a94d897d5edeef4bf8f9aa3f2f284482
Parents: ab21cdf
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Mon May 22 16:30:59 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:53:15 2017 -0700
----------------------------------------------------------------------
LICENSE | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/b082dd80/LICENSE
----------------------------------------------------------------------
diff --git a/LICENSE b/LICENSE
index 86a58c7..1327411 100644
--- a/LICENSE
+++ b/LICENSE
@@ -648,3 +648,38 @@ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
+
+
+======================================================================
+For grpc-1.4.2
+(3rdparty/grpc-1.4.2.tar.gz):
+======================================================================
+
+Copyright 2015, Google Inc.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+ * Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
[07/11] mesos git commit: Added unit tests for gRPC support in
libprocess.
Posted by ji...@apache.org.
Added unit tests for gRPC support in libprocess.
We tested the following 6 scenarios when making gRPC calls:
1. The server responds with an OK.
2. The server responds with a failure.
3. The client discards the call before the server starts.
4. The client discards the call when the server is processing it.
5. The client makes mulitple concurrent calls.
6. The client shuts down during a call.
7. The server is unreachable.
8. The server dose not respond in time.
Review: https://reviews.apache.org/r/61098
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/f2b16cab
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/f2b16cab
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/f2b16cab
Branch: refs/heads/master
Commit: f2b16cab5449a3a6def8cb16a88d2252984c9cfa
Parents: 7ce4053
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Mon Jul 24 17:05:56 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:05 2017 -0700
----------------------------------------------------------------------
3rdparty/libprocess/src/tests/grpc_tests.cpp | 367 ++++++++++++++++++++
3rdparty/libprocess/src/tests/grpc_tests.proto | 35 ++
2 files changed, 402 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/f2b16cab/3rdparty/libprocess/src/tests/grpc_tests.cpp
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/src/tests/grpc_tests.cpp b/3rdparty/libprocess/src/tests/grpc_tests.cpp
new file mode 100644
index 0000000..a11b6f0
--- /dev/null
+++ b/3rdparty/libprocess/src/tests/grpc_tests.cpp
@@ -0,0 +1,367 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <string>
+
+#include <gmock/gmock.h>
+
+#include <process/future.hpp>
+#include <process/grpc.hpp>
+#include <process/gtest.hpp>
+
+#include <stout/path.hpp>
+#include <stout/try.hpp>
+
+#include <stout/tests/utils.hpp>
+
+#include "grpc_tests.pb.h"
+#include "grpc_tests.grpc.pb.h"
+
+namespace client = process::grpc::client;
+
+using std::shared_ptr;
+using std::string;
+using std::unique_ptr;
+
+using ::grpc::InsecureServerCredentials;
+using ::grpc::Server;
+using ::grpc::ServerBuilder;
+using ::grpc::ServerContext;
+using ::grpc::Status;
+
+using process::Future;
+using process::Promise;
+
+using process::grpc::Channel;
+
+using testing::_;
+using testing::DoAll;
+using testing::InvokeWithoutArgs;
+using testing::Return;
+
+namespace tests {
+
+class PingPongServer : public PingPong::Service
+{
+public:
+ PingPongServer()
+ {
+ EXPECT_CALL(*this, Send(_, _, _))
+ .WillRepeatedly(Return(Status::OK));
+ }
+
+ MOCK_METHOD3(Send, Status(ServerContext*, const Ping* ping, Pong* pong));
+
+ Try<Nothing> Startup(const string& address)
+ {
+ ServerBuilder builder;
+ builder.AddListeningPort(address, InsecureServerCredentials());
+ builder.RegisterService(this);
+
+ server = builder.BuildAndStart();
+ if (!server) {
+ return Error("Unable to start a gRPC server.");
+ }
+
+ return Nothing();
+ }
+
+ Try<Nothing> Shutdown()
+ {
+ server->Shutdown();
+ server->Wait();
+
+ return Nothing();
+ }
+
+private:
+ unique_ptr<Server> server;
+};
+
+
+class GRPCClientTest : public TemporaryDirectoryTest {
+protected:
+ string server_address() const
+ {
+ // TODO(chhsiao): Use in-process tranport instead of a Unix domain
+ // socket once gRPC supports it for Windows support.
+ // https://github.com/grpc/grpc/pull/11145
+ return "unix://" + path::join(sandbox.get(), "socket");
+ }
+};
+
+
+// This test verifies that a gRPC future is properly set once the
+// given call is processed by the server.
+TEST_F(GRPCClientTest, Success)
+{
+ PingPongServer server;
+ ASSERT_SOME(server.Startup(server_address()));
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+
+ AWAIT_EXPECT_READY(pong);
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+
+// This test verifies that we can have multiple outstanding gRPC calls.
+TEST_F(GRPCClientTest, ConcurrentRPCs)
+{
+ PingPongServer server;
+ ASSERT_SOME(server.Startup(server_address()));
+
+ shared_ptr<Promise<Nothing>> processed1(new Promise<Nothing>());
+ shared_ptr<Promise<Nothing>> processed2(new Promise<Nothing>());
+ shared_ptr<Promise<Nothing>> processed3(new Promise<Nothing>());
+ shared_ptr<Promise<Nothing>> pinged(new Promise<Nothing>());
+
+ EXPECT_CALL(server, Send(_, _, _))
+ .WillOnce(DoAll(
+ InvokeWithoutArgs([=] {
+ processed1->set(Nothing());
+ AWAIT_READY(pinged->future());
+ }),
+ Return(Status::OK)))
+ .WillOnce(DoAll(
+ InvokeWithoutArgs([=] {
+ processed2->set(Nothing());
+ AWAIT_READY(pinged->future());
+ }),
+ Return(Status::OK)))
+ .WillOnce(DoAll(
+ InvokeWithoutArgs([=] {
+ processed3->set(Nothing());
+ AWAIT_READY(pinged->future());
+ }),
+ Return(Status::OK)));
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ Future<Pong> pong1 = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+ Future<Pong> pong2 = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+ Future<Pong> pong3 = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+
+ AWAIT_READY(processed1->future());
+ AWAIT_READY(processed2->future());
+ AWAIT_READY(processed3->future());
+
+ pinged->set(Nothing());
+
+ AWAIT_EXPECT_READY(pong1);
+ AWAIT_EXPECT_READY(pong2);
+ AWAIT_EXPECT_READY(pong3);
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+
+// This test verifies that a gRPC future fails when the server responds
+// with a status other than OK for the given call.
+TEST_F(GRPCClientTest, Failed)
+{
+ PingPongServer server;
+
+ EXPECT_CALL(server, Send(_, _, _))
+ .WillOnce(Return(Status::CANCELLED));
+
+ ASSERT_SOME(server.Startup(server_address()));
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+
+ AWAIT_EXPECT_FAILED(pong);
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+
+// This test verifies that a gRPC future can be discarded before the
+// server processes the given call.
+TEST_F(GRPCClientTest, DiscardedBeforeServerStarted)
+{
+ PingPongServer server;
+
+ EXPECT_CALL(server, Send(_, _, _))
+ .Times(0);
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+ pong.discard();
+
+ ASSERT_SOME(server.Startup(server_address()));
+
+ AWAIT_EXPECT_DISCARDED(pong);
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+
+// This test verifies that a gRPC future can be discarded before and
+// when the server is processing the given call.
+TEST_F(GRPCClientTest, DiscardedWhenServerProcessing)
+{
+ PingPongServer server;
+
+ shared_ptr<Promise<Nothing>> processed(new Promise<Nothing>());
+ shared_ptr<Promise<Nothing>> discarded(new Promise<Nothing>());
+
+ EXPECT_CALL(server, Send(_, _, _))
+ .WillOnce(DoAll(
+ InvokeWithoutArgs([=] {
+ processed->set(Nothing());
+ AWAIT_READY(discarded->future());
+ }),
+ Return(Status::OK)));
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ ASSERT_SOME(server.Startup(server_address()));
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+ AWAIT_READY(processed->future());
+
+ pong.discard();
+ discarded->set(Nothing());
+
+ AWAIT_EXPECT_DISCARDED(pong);
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+
+// This test verifies that a gRPC future fails properly when the runtime
+// is shut down before the server responds.
+TEST_F(GRPCClientTest, ClientShutdown)
+{
+ PingPongServer server;
+
+ shared_ptr<Promise<Nothing>> processed(new Promise<Nothing>());
+ shared_ptr<Promise<Nothing>> shutdown(new Promise<Nothing>());
+
+ EXPECT_CALL(server, Send(_, _, _))
+ .WillOnce(DoAll(
+ InvokeWithoutArgs([=] {
+ processed->set(Nothing());
+ AWAIT_READY(shutdown->future());
+ }),
+ Return(Status::OK)));
+
+ ASSERT_SOME(server.Startup(server_address()));
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+
+ AWAIT_READY(processed->future());
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ shutdown->set(Nothing());
+ AWAIT_EXPECT_FAILED(pong);
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+
+// This test verifies that a gRPC future fails when it is unable to
+// connect to the server.
+TEST_F(GRPCClientTest, ServerUnreachable)
+{
+ client::Runtime runtime;
+ Channel channel("nosuchhost");
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ AWAIT_EXPECT_FAILED(pong);
+}
+
+
+// This test verifies that a gRPC future fails properly when the server
+// hangs when processing the given call.
+TEST_F(GRPCClientTest, ServerTimeout)
+{
+ PingPongServer server;
+
+ shared_ptr<Promise<Nothing>> done(new Promise<Nothing>());
+
+ EXPECT_CALL(server, Send(_, _, _))
+ .WillOnce(DoAll(
+ InvokeWithoutArgs([=] {
+ AWAIT_READY(done->future());
+ }),
+ Return(Status::OK)));
+
+ ASSERT_SOME(server.Startup(server_address()));
+
+ client::Runtime runtime;
+ Channel channel(server_address());
+
+ Future<Pong> pong = runtime.call(
+ channel, GRPC_RPC(PingPong, Send), Ping());
+
+ // TODO(chhsiao): The gRPC library returns a failure after the default
+ // timeout (5 seconds) is passed, no matter when the `CompletionQueue`
+ // is shut down. The timeout should be lowered once we support it.
+ AWAIT_EXPECT_FAILED(pong);
+ done->set(Nothing());
+
+ runtime.terminate();
+ AWAIT_ASSERT_READY(runtime.wait());
+
+ ASSERT_SOME(server.Shutdown());
+}
+
+} // namespace tests {
http://git-wip-us.apache.org/repos/asf/mesos/blob/f2b16cab/3rdparty/libprocess/src/tests/grpc_tests.proto
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/src/tests/grpc_tests.proto b/3rdparty/libprocess/src/tests/grpc_tests.proto
new file mode 100644
index 0000000..c461842
--- /dev/null
+++ b/3rdparty/libprocess/src/tests/grpc_tests.proto
@@ -0,0 +1,35 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package tests;
+
+// NOTE: The generated headers for this file have been included in the
+// tests folder to simplify the build process (no need to have protoc
+// available to compile this file). As a result, if there are any
+// changes to this file, the headers must be re-generated and committed
+// alongside changes to this file.
+
+message Ping {}
+
+
+message Pong {}
+
+
+service PingPong {
+ rpc Send(Ping) returns (Pong) {}
+}
[06/11] mesos git commit: Added a patch for building/installing the
bundled gRPC library.
Posted by ji...@apache.org.
Added a patch for building/installing the bundled gRPC library.
This patch is cherry-picked from the following 2 commits in gRPC's
master branch and 2 PR's with at least one approval:
https://github.com/grpc/grpc/commit/807693b
https://github.com/grpc/grpc/commit/43f7acf
https://github.com/grpc/grpc/pull/11558
https://github.com/grpc/grpc/pull/11565
Review: https://reviews.apache.org/r/61576
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/70a41097
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/70a41097
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/70a41097
Branch: refs/heads/master
Commit: 70a410979508a5f09cf6c13f6bb7b77dd99f3292
Parents: b082dd8
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Thu Aug 10 12:02:26 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:05 2017 -0700
----------------------------------------------------------------------
3rdparty/grpc-1.4.2.patch | 178 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 178 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/70a41097/3rdparty/grpc-1.4.2.patch
----------------------------------------------------------------------
diff --git a/3rdparty/grpc-1.4.2.patch b/3rdparty/grpc-1.4.2.patch
new file mode 100644
index 0000000..bce9871
--- /dev/null
+++ b/3rdparty/grpc-1.4.2.patch
@@ -0,0 +1,178 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index ac6804a7a8..27c40f0985 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -1504,6 +1504,8 @@ target_include_directories(grpc_cronet
+ target_link_libraries(grpc_cronet
+ ${_gRPC_BASELIB_LIBRARIES}
+ ${_gRPC_SSL_LIBRARIES}
++ ${_gRPC_ZLIB_LIBRARIES}
++ ${_gRPC_CARES_LIBRARIES}
+ ${_gRPC_ALLTARGETS_LIBRARIES}
+ gpr
+ )
+@@ -2094,6 +2096,8 @@ target_include_directories(grpc_unsecure
+
+ target_link_libraries(grpc_unsecure
+ ${_gRPC_BASELIB_LIBRARIES}
++ ${_gRPC_ZLIB_LIBRARIES}
++ ${_gRPC_CARES_LIBRARIES}
+ ${_gRPC_ALLTARGETS_LIBRARIES}
+ gpr
+ )
+diff --git a/Makefile b/Makefile
+index 3d2466fce9..8ac4dec61c 100644
+--- a/Makefile
++++ b/Makefile
+@@ -2650,7 +2650,7 @@ install-shared_cxx: shared_cxx strip-shared_cxx install-shared_c install-pkg-con
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/libgrpc++$(SHARED_VERSION_CPP)-dll.a $(prefix)/lib/libgrpc++.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)grpc++$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++.so.4
++ $(Q) ln -sf $(SHARED_PREFIX)grpc++$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++.so.1
+ $(Q) ln -sf $(SHARED_PREFIX)grpc++$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++.so
+ endif
+ $(E) "[INSTALL] Installing $(SHARED_PREFIX)grpc++_cronet$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP)"
+@@ -2659,7 +2659,7 @@ endif
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/libgrpc++_cronet$(SHARED_VERSION_CPP)-dll.a $(prefix)/lib/libgrpc++_cronet.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)grpc++_cronet$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_cronet.so.4
++ $(Q) ln -sf $(SHARED_PREFIX)grpc++_cronet$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_cronet.so.1
+ $(Q) ln -sf $(SHARED_PREFIX)grpc++_cronet$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_cronet.so
+ endif
+ $(E) "[INSTALL] Installing $(SHARED_PREFIX)grpc++_error_details$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP)"
+@@ -2668,7 +2668,7 @@ endif
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/libgrpc++_error_details$(SHARED_VERSION_CPP)-dll.a $(prefix)/lib/libgrpc++_error_details.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)grpc++_error_details$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_error_details.so.4
++ $(Q) ln -sf $(SHARED_PREFIX)grpc++_error_details$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_error_details.so.1
+ $(Q) ln -sf $(SHARED_PREFIX)grpc++_error_details$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_error_details.so
+ endif
+ $(E) "[INSTALL] Installing $(SHARED_PREFIX)grpc++_reflection$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP)"
+@@ -2677,7 +2677,7 @@ endif
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/libgrpc++_reflection$(SHARED_VERSION_CPP)-dll.a $(prefix)/lib/libgrpc++_reflection.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)grpc++_reflection$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_reflection.so.4
++ $(Q) ln -sf $(SHARED_PREFIX)grpc++_reflection$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_reflection.so.1
+ $(Q) ln -sf $(SHARED_PREFIX)grpc++_reflection$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_reflection.so
+ endif
+ $(E) "[INSTALL] Installing $(SHARED_PREFIX)grpc++_unsecure$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP)"
+@@ -2686,7 +2686,7 @@ endif
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/libgrpc++_unsecure$(SHARED_VERSION_CPP)-dll.a $(prefix)/lib/libgrpc++_unsecure.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)grpc++_unsecure$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_unsecure.so.4
++ $(Q) ln -sf $(SHARED_PREFIX)grpc++_unsecure$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_unsecure.so.1
+ $(Q) ln -sf $(SHARED_PREFIX)grpc++_unsecure$(SHARED_VERSION_CPP).$(SHARED_EXT_CPP) $(prefix)/lib/libgrpc++_unsecure.so
+ endif
+ ifneq ($(SYSTEM),MINGW32)
+@@ -2703,7 +2703,7 @@ install-shared_csharp: shared_csharp strip-shared_csharp
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/libgrpc_csharp_ext$(SHARED_VERSION_CSHARP)-dll.a $(prefix)/lib/libgrpc_csharp_ext.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)grpc_csharp_ext$(SHARED_VERSION_CSHARP).$(SHARED_EXT_CSHARP) $(prefix)/lib/libgrpc_csharp_ext.so.4
++ $(Q) ln -sf $(SHARED_PREFIX)grpc_csharp_ext$(SHARED_VERSION_CSHARP).$(SHARED_EXT_CSHARP) $(prefix)/lib/libgrpc_csharp_ext.so.1
+ $(Q) ln -sf $(SHARED_PREFIX)grpc_csharp_ext$(SHARED_VERSION_CSHARP).$(SHARED_EXT_CSHARP) $(prefix)/lib/libgrpc_csharp_ext.so
+ endif
+ ifneq ($(SYSTEM),MINGW32)
+diff --git a/src/c-ares/CMakeLists.txt b/src/c-ares/CMakeLists.txt
+index 22acd51048..a5add7a64c 100644
+--- a/src/c-ares/CMakeLists.txt
++++ b/src/c-ares/CMakeLists.txt
+@@ -45,5 +45,53 @@ else()
+ add_definitions(-D_GNU_SOURCE=1)
+ endif()
+
+-file(GLOB lib_sources ../../third_party/cares/cares/*.c)
+-add_library(cares ${lib_sources})
++add_library(cares
++ ../../third_party/cares/cares/ares__close_sockets.c
++ ../../third_party/cares/cares/ares__get_hostent.c
++ ../../third_party/cares/cares/ares__read_line.c
++ ../../third_party/cares/cares/ares__timeval.c
++ ../../third_party/cares/cares/ares_cancel.c
++ ../../third_party/cares/cares/ares_data.c
++ ../../third_party/cares/cares/ares_destroy.c
++ ../../third_party/cares/cares/ares_expand_name.c
++ ../../third_party/cares/cares/ares_expand_string.c
++ ../../third_party/cares/cares/ares_fds.c
++ ../../third_party/cares/cares/ares_free_hostent.c
++ ../../third_party/cares/cares/ares_free_string.c
++ ../../third_party/cares/cares/ares_getenv.c
++ ../../third_party/cares/cares/ares_gethostbyaddr.c
++ ../../third_party/cares/cares/ares_gethostbyname.c
++ ../../third_party/cares/cares/ares_getnameinfo.c
++ ../../third_party/cares/cares/ares_getsock.c
++ ../../third_party/cares/cares/ares_init.c
++ ../../third_party/cares/cares/ares_library_init.c
++ ../../third_party/cares/cares/ares_llist.c
++ ../../third_party/cares/cares/ares_mkquery.c
++ ../../third_party/cares/cares/ares_create_query.c
++ ../../third_party/cares/cares/ares_nowarn.c
++ ../../third_party/cares/cares/ares_options.c
++ ../../third_party/cares/cares/ares_parse_a_reply.c
++ ../../third_party/cares/cares/ares_parse_aaaa_reply.c
++ ../../third_party/cares/cares/ares_parse_mx_reply.c
++ ../../third_party/cares/cares/ares_parse_naptr_reply.c
++ ../../third_party/cares/cares/ares_parse_ns_reply.c
++ ../../third_party/cares/cares/ares_parse_ptr_reply.c
++ ../../third_party/cares/cares/ares_parse_soa_reply.c
++ ../../third_party/cares/cares/ares_parse_srv_reply.c
++ ../../third_party/cares/cares/ares_parse_txt_reply.c
++ ../../third_party/cares/cares/ares_platform.c
++ ../../third_party/cares/cares/ares_process.c
++ ../../third_party/cares/cares/ares_query.c
++ ../../third_party/cares/cares/ares_search.c
++ ../../third_party/cares/cares/ares_send.c
++ ../../third_party/cares/cares/ares_strcasecmp.c
++ ../../third_party/cares/cares/ares_strdup.c
++ ../../third_party/cares/cares/ares_strerror.c
++ ../../third_party/cares/cares/ares_timeout.c
++ ../../third_party/cares/cares/ares_version.c
++ ../../third_party/cares/cares/ares_writev.c
++ ../../third_party/cares/cares/bitncmp.c
++ ../../third_party/cares/cares/inet_net_pton.c
++ ../../third_party/cares/cares/inet_ntop.c
++ ../../third_party/cares/cares/windows_port.c
++)
+diff --git a/src/core/ext/census/tracing.c b/src/core/ext/census/tracing.c
+index 8b74628f4f..6bbeca27d9 100644
+--- a/src/core/ext/census/tracing.c
++++ b/src/core/ext/census/tracing.c
+@@ -36,7 +36,6 @@
+ #include <grpc/census.h>
+ #include <grpc/support/alloc.h>
+ #include <grpc/support/log.h>
+-#include <openssl/rand.h>
+ #include "src/core/ext/census/mlog.h"
+
+ void trace_start_span(const trace_span_context *span_ctxt,
+diff --git a/templates/CMakeLists.txt.template b/templates/CMakeLists.txt.template
+index 2252a7ea80..82ea9477c3 100644
+--- a/templates/CMakeLists.txt.template
++++ b/templates/CMakeLists.txt.template
+@@ -58,7 +58,7 @@
+ deps.append("${_gRPC_SSL_LIBRARIES}")
+ if target_dict.language == 'c++':
+ deps.append("${_gRPC_PROTOBUF_LIBRARIES}")
+- if target_dict['name'] in ['grpc']:
++ if target_dict['name'] in ['grpc', 'grpc_cronet', 'grpc_unsecure']:
+ deps.append("${_gRPC_ZLIB_LIBRARIES}")
+ deps.append("${_gRPC_CARES_LIBRARIES}")
+ deps.append("${_gRPC_ALLTARGETS_LIBRARIES}")
+diff --git a/templates/Makefile.template b/templates/Makefile.template
+index a9dd171b84..2c638da3c8 100644
+--- a/templates/Makefile.template
++++ b/templates/Makefile.template
+@@ -1349,7 +1349,7 @@
+ ifeq ($(SYSTEM),MINGW32)
+ $(Q) $(INSTALL) $(LIBDIR)/$(CONFIG)/lib${lib.name}$(SHARED_VERSION_${lang_to_var[lib.language]})-dll.a $(prefix)/lib/lib${lib.name}.a
+ else ifneq ($(SYSTEM),Darwin)
+- $(Q) ln -sf $(SHARED_PREFIX)${lib.name}$(SHARED_VERSION_${lang_to_var[lib.language]}).$(SHARED_EXT_${lang_to_var[lib.language]}) $(prefix)/lib/lib${lib.name}.so.${settings.core_version.major}
++ $(Q) ln -sf $(SHARED_PREFIX)${lib.name}$(SHARED_VERSION_${lang_to_var[lib.language]}).$(SHARED_EXT_${lang_to_var[lib.language]}) $(prefix)/lib/lib${lib.name}.so.${settings.get(lang_to_var[lib.language].lower() + '_version').major}
+ $(Q) ln -sf $(SHARED_PREFIX)${lib.name}$(SHARED_VERSION_${lang_to_var[lib.language]}).$(SHARED_EXT_${lang_to_var[lib.language]}) $(prefix)/lib/lib${lib.name}.so
+ endif
+ % endif
[04/11] mesos git commit: Removed `--disable-zlib` and fixed
`--with-zlib` for Mesos.
Posted by ji...@apache.org.
Removed `--disable-zlib` and fixed `--with-zlib` for Mesos.
For third-party libraries that does not support `--with-zlib=DIR`, we
introduce new variables `ZLIB_CPPFLAGS` and `ZLIB_LINKERFLAGS` so that
they can be used to set up `CPPFLAGS` and `LDFLAGS` when building those
libraries.
Review: https://reviews.apache.org/r/61508
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/30914ea9
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/30914ea9
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/30914ea9
Branch: refs/heads/master
Commit: 30914ea9445e2ec3eb48e2daad814accca8f404c
Parents: b9e27f2
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Tue Aug 8 13:39:33 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:53:15 2017 -0700
----------------------------------------------------------------------
3rdparty/Makefile.am | 15 +++++++++++++--
configure.ac | 42 ++++++++++++++++++------------------------
2 files changed, 31 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/30914ea9/3rdparty/Makefile.am
----------------------------------------------------------------------
diff --git a/3rdparty/Makefile.am b/3rdparty/Makefile.am
index ce32885..742b8bf 100644
--- a/3rdparty/Makefile.am
+++ b/3rdparty/Makefile.am
@@ -37,6 +37,10 @@ CONFIGURE_ARGS = @CONFIGURE_ARGS@ --enable-shared=no --with-pic --srcdir=.
# `make install DESTDIR=${MESOS-PREFIX}/lib/mesos/3rdparty install`
CONFIGURE_ARGS += --prefix=/
+# Set up building flags for non-bundled libraries.
+ZLIB_INCLUDE_FLAGS = @ZLIB_CPPFLAGS@
+ZLIB_LINKER_FLAGS = @ZLIB_LINKERFLAGS@
+
# Directory where bundled 3rdparty module dependencies are installed.
pkg3rdpartydir = $(pkglibdir)/3rdparty
@@ -263,9 +267,16 @@ endif
if WITH_BUNDLED_PROTOBUF
$(PROTOBUF)/src/protoc $(PROTOBUF)/src/libprotobuf.la: $(PROTOBUF)-build-stamp
+# NOTE: The `--with-zlib` flag works differently between Mesos and
+# Protobuf. Protobuf uses `--with-zlib` as an on/off switch for data
+# compression instead of a path specifier for ZLib, so we need to set up
+# the `CPPFLAGS` and `LDFLAGS` if `--with-zlib=DIR` is given.
$(PROTOBUF)-build-stamp: $(PROTOBUF)-stamp
- cd $(PROTOBUF) && ./configure $(CONFIGURE_ARGS) && \
- $(MAKE) $(AM_MAKEFLAGS)
+ cd $(PROTOBUF) && \
+ ./configure $(CONFIGURE_ARGS) \
+ CPPFLAGS="$(ZLIB_INCLUDE_FLAGS)" \
+ LDFLAGS="$(ZLIB_LINKER_FLAGS)" && \
+ $(MAKE) $(AM_MAKEFLAGS)
touch $@
ALL_LOCAL += $(PROTOBUF)/src/libprotobuf.la
ALL_LOCAL += $(PROTOBUF)/src/protoc
http://git-wip-us.apache.org/repos/asf/mesos/blob/30914ea9/configure.ac
----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index 307f0ae..08f4339 100644
--- a/configure.ac
+++ b/configure.ac
@@ -273,12 +273,6 @@ AC_ARG_ENABLE([xfs-disk-isolator],
[builds the XFS disk isolator]),
[], [enable_xfs_disk_isolator=no])
-AC_ARG_ENABLE([zlib],
- AS_HELP_STRING([--disable-zlib],
- [disables zlib compression, which means the webui
- will be far less responsive; not recommended]),
- [], [enable_zlib=yes])
-
AC_ARG_ENABLE([parallel_test_execution],
AS_HELP_STRING([--enable-parallel-test-execution],
[execute tests in parallel where possible]))
@@ -705,13 +699,6 @@ else
GFLAGS_LIBS=""
fi
-AC_CHECK_LIB([z], [gzread], [],
- [AC_MSG_ERROR([cannot find libz
--------------------------------------------------------------------
-libz is required for mesos to build.
--------------------------------------------------------------------
-])])
-
AC_CHECK_TOOL([PATCHCOMMAND_TEST], [patch], [], [$PATH])
if test -z "`echo $PATCHCOMMAND_TEST`"; then
AC_MSG_ERROR([patch command not found in PATH])
@@ -1913,21 +1900,28 @@ AM_CONDITIONAL([ENABLE_XFS_DISK_ISOLATOR], [test "x$enable_xfs_disk_isolator" =
# the CPPFLAGS and LDFLAGS with respective /include and /lib path
# suffixes.
if test -n "`echo $with_zlib`" ; then
- CPPFLAGS="-isystem ${with_zlib}/include $CPPFLAGS"
- LDFLAGS="-L${with_zlib}/lib $LDFLAGS"
+ ZLIB_CPPFLAGS="-isystem ${with_zlib}/include"
+ ZLIB_LINKERFLAGS="-L${with_zlib}/lib"
+
+ CPPFLAGS="$ZLIB_CPPFLAGS $CPPFLAGS"
+ LDFLAGS="$ZLIB_LINKERFLAGS $LDFLAGS"
fi
-# Check if we should/can build with libz.
-if test "x$enable_zlib" = "xyes"; then
- AC_CHECK_LIB([z], [deflate, gzread, gzwrite, inflate], [],
- [AC_MSG_ERROR([cannot find libz
+AC_CHECK_HEADERS([zlib.h],
+ [AC_CHECK_LIB([z], [deflate, gzread, gzwrite, inflate], [],
+ [AC_MSG_ERROR([cannot find libz
-------------------------------------------------------------------
-This means HTTP responses will be slower because we cannot use
-compression; you probably want to download and install zlib, but
-you can get away without it by doing --disable-zlib.
+libz is required for Mesos to build.
-------------------------------------------------------------------
- ])])
-fi
+ ])])],
+ [AC_MSG_ERROR([cannot find libz headers
+-------------------------------------------------------------------
+libz headers are required for Mesos to build.
+-------------------------------------------------------------------
+ ])])
+
+AC_SUBST([ZLIB_CPPFLAGS])
+AC_SUBST([ZLIB_LINKERFLAGS])
# Check if ZooKeeper prefix path was supplied and if so, add it to
[08/11] mesos git commit: Refactored OpenSSL library checks in
libprocess.
Posted by ji...@apache.org.
Refactored OpenSSL library checks in libprocess.
Since gRPC requires OpenSSL, we checks if libssl and libcrypto exist no
matter if the `--enable-ssl` flag is on. This enables us to have a
non-SSL-enabled libprocess build with gRPC support. Also fixed an error
that a bad linker might link a configure test with libssl unnecessarily,
which would cause a runtime failure if libssl is not in the runtime
library search path.
Review: https://reviews.apache.org/r/61517
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/fa0494e3
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/fa0494e3
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/fa0494e3
Branch: refs/heads/master
Commit: fa0494e3ebadb40637c27a5581fd961060a20f93
Parents: fa8b311
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Tue Aug 8 19:17:39 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:06 2017 -0700
----------------------------------------------------------------------
3rdparty/libprocess/configure.ac | 37 +++++++++++++++++++++++------------
1 file changed, 24 insertions(+), 13 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/fa0494e3/3rdparty/libprocess/configure.ac
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/configure.ac b/3rdparty/libprocess/configure.ac
index db45de0..31556f7 100644
--- a/3rdparty/libprocess/configure.ac
+++ b/3rdparty/libprocess/configure.ac
@@ -952,27 +952,33 @@ if test -n "`echo $with_ssl`"; then
LDFLAGS="-L${with_ssl}/lib $LDFLAGS"
fi
+AC_CHECK_LIB([crypto], [RAND_poll], [found_crypto=yes])
+
+AC_CHECK_HEADERS([openssl/ssl.h],
+ [AC_CHECK_LIB([ssl],
+ [SSL_CTX_new],
+ [found_ssl=yes],
+ [],
+ [-lcrypto])])
+
if test "x$enable_ssl" = "xyes"; then
- AC_CHECK_HEADERS([openssl/ssl.h],
- [AC_CHECK_LIB([ssl],
- [SSL_CTX_new],
- [],
- [AC_MSG_ERROR([cannot find libssl
--------------------------------------------------------------------
-libssl is required for an SSL-enabled build.
--------------------------------------------------------------------
- ])])],
- [AC_MSG_ERROR([cannot find libssl headers
+ if test "x$found_ssl" != "xyes"; then
+ AC_MSG_ERROR([cannot find libssl
-------------------------------------------------------------------
libssl is required for an SSL-enabled build.
-------------------------------------------------------------------
- ])])
+ ])
+ fi
- AC_CHECK_LIB([crypto], [RAND_poll], [], [AC_MSG_ERROR([cannot find libcrypto
+ if test "x$found_crypto" != "xyes"; then
+ AC_MSG_ERROR([cannot find libcrypto
-------------------------------------------------------------------
libcrypto is required for an SSL-enabled build.
-------------------------------------------------------------------
- ])])
+ ])
+ fi
+
+ LIBS="-lssl -lcrypto $LIBS"
if test "x$enable_libevent" = "xyes"; then
AC_CHECK_HEADERS([event2/bufferevent_ssl.h],
@@ -1106,6 +1112,10 @@ AC_SUBST([ZLIB_LINKERFLAGS])
# affected by libcxx undefined behavior,
# https://llvm.org/bugs/show_bug.cgi?id=28469.
AC_MSG_CHECKING([C++ standard library for undefined behaviour with selected optimization level])
+# NOTE: We clear `LIBS` here to prevent linking in libraries unrelated
+# to the test. These libraries might not be in the linker lookup paths.
+saved_LIBS="$LIBS"
+LIBS=""
AC_LANG_PUSH([C++])
AC_RUN_IFELSE([
AC_LANG_SOURCE([[
@@ -1131,6 +1141,7 @@ AC_RUN_IFELSE([
]
)
AC_LANG_POP([C++])
+LIBS="$saved_LIBS"
# Also pass the flags to 3rdparty libraries.
[10/11] mesos git commit: Building gRPC with Autotools.
Posted by ji...@apache.org.
Building gRPC with Autotools.
This patch enables building gRPC through Autotools under Linux with
opetion `--enable-grpc`. Additionally, another option `--enable-grpc` is
added to let the user specify the path to a custom gRPC library. This
option depends on `--with-protobuf` to avoid using conflicting versions
of Protobuf libraries.
For OpenSSL and unbundled Protobuf, we introduce new variables
`<lib>_CPPFLAGS` and `<lib>_LINKERFLAGS` for building gRPC.
Review: https://reviews.apache.org/r/61159
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/90f7a7e1
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/90f7a7e1
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/90f7a7e1
Branch: refs/heads/master
Commit: 90f7a7e140d4dbee53a0edc2ab225a4ea83db3a5
Parents: fa0494e
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Tue May 23 10:49:00 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:06 2017 -0700
----------------------------------------------------------------------
3rdparty/Makefile.am | 50 +++++++++++++++++++
3rdparty/versions.am | 1 +
configure.ac | 123 ++++++++++++++++++++++++++++++++++++++++++++--
src/Makefile.am | 15 ++++++
4 files changed, 185 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/90f7a7e1/3rdparty/Makefile.am
----------------------------------------------------------------------
diff --git a/3rdparty/Makefile.am b/3rdparty/Makefile.am
index 742b8bf..2182837 100644
--- a/3rdparty/Makefile.am
+++ b/3rdparty/Makefile.am
@@ -38,6 +38,8 @@ CONFIGURE_ARGS = @CONFIGURE_ARGS@ --enable-shared=no --with-pic --srcdir=.
CONFIGURE_ARGS += --prefix=/
# Set up building flags for non-bundled libraries.
+SSL_INCLUDE_FLAGS = @SSL_CPPFLAGS@
+SSL_LINKER_FLAGS = @SSL_LINKERFLAGS@
ZLIB_INCLUDE_FLAGS = @ZLIB_CPPFLAGS@
ZLIB_LINKER_FLAGS = @ZLIB_LINKERFLAGS@
@@ -53,6 +55,7 @@ GLOG = glog-$(GLOG_VERSION)
GOOGLETEST = googletest-release-$(GOOGLETEST_VERSION)
GMOCK = $(GOOGLETEST)/googlemock
GPERFTOOLS = gperftools-$(GPERFTOOLS_VERSION)
+GRPC = grpc-$(GRPC_VERSION)
GTEST = $(GOOGLETEST)/googletest
HTTP_PARSER = http-parser-$(HTTP_PARSER_VERSION)
LEVELDB = leveldb-$(LEVELDB_VERSION)
@@ -72,6 +75,7 @@ EXTRA_DIST = \
$(GLOG).tar.gz \
$(GOOGLETEST).tar.gz \
$(GPERFTOOLS).tar.gz \
+ $(GRPC).tar.gz \
$(HTTP_PARSER).tar.gz \
$(LEVELDB).tar.gz \
$(LIBEV).tar.gz \
@@ -88,6 +92,10 @@ EXTRA_DIST = \
EXTRA_DIST += \
$(GLOG).patch
+# We patch grpc for CMake compilation issues.
+EXTRA_DIST += \
+ $(GRPC).patch
+
# We need to patch leveldb to prevent it from linking to snappy and
# tcmalloc; see MESOS-1054, MESOS-1624.
EXTRA_DIST += \
@@ -108,6 +116,7 @@ CLEAN_EXTRACTED = \
$(GLOG) \
$(GOOGLETEST) \
$(GPERFTOOLS) \
+ $(GRPC) \
$(HTTP_PARSER) \
$(LEVELDB) \
$(LIBEV) \
@@ -280,6 +289,47 @@ $(PROTOBUF)-build-stamp: $(PROTOBUF)-stamp
touch $@
ALL_LOCAL += $(PROTOBUF)/src/libprotobuf.la
ALL_LOCAL += $(PROTOBUF)/src/protoc
+
+PROTOBUF_INCLUDE_FLAGS = -I$(abs_builddir)/$(PROTOBUF)/src
+PROTOBUF_LINKER_FLAGS = -L$(abs_builddir)/$(PROTOBUF)/src/.libs
+PROTOC = $(abs_builddir)/$(PROTOBUF)/src/protoc
+else
+$(PROTOBUF)-build-stamp:
+ touch $@
+
+PROTOBUF_INCLUDE_FLAGS = @PROTOBUF_CPPFLAGS@
+PROTOBUF_LINKER_FLAGS = @PROTOBUF_LINKERFLAGS@
+PROTOC = @PROTOCOMPILER@
+endif
+
+if ENABLE_GRPC
+if WITH_BUNDLED_GRPC
+LIB_GRPC = $(GRPC)/libs/opt/libgrpc++.a \
+ $(GRPC)/libs/opt/libgrpc.a \
+ $(GRPC)/libs/opt/libgpr.a
+
+$(LIB_GRPC): $(GRPC)-build-stamp
+
+$(GRPC)-build-stamp: $(GRPC)-stamp \
+ $(PROTOBUF)-build-stamp
+ cd $(GRPC) && \
+ CPPFLAGS="$(PROTOBUF_INCLUDE_FLAGS) \
+ $(SSL_INCLUDE_FLAGS) \
+ $(ZLIB_INCLUDE_FLAGS)" \
+ LDFLAGS="$(PROTOBUF_LINKER_FLAGS) \
+ $(SSL_LINKER_FLAGS) \
+ $(ZLIB_LINKER_FLAGS)" \
+ $(MAKE) $(AM_MAKEFLAGS) \
+ HAS_PKG_CONFIG=false \
+ NO_PROTOC=false \
+ PROTOC=$(PROTOC)
+ touch $@
+
+ALL_LOCAL += $(LIB_GRPC)
+else
+$(GRPC)-build-stamp:
+ touch $@
+endif
endif
# Convenience library for gmock/gtest.
http://git-wip-us.apache.org/repos/asf/mesos/blob/90f7a7e1/3rdparty/versions.am
----------------------------------------------------------------------
diff --git a/3rdparty/versions.am b/3rdparty/versions.am
index 0893572..82d66be 100644
--- a/3rdparty/versions.am
+++ b/3rdparty/versions.am
@@ -25,6 +25,7 @@ ELFIO_VERSION = 3.2
GLOG_VERSION = 0.3.3
GOOGLETEST_VERSION = 1.8.0
GPERFTOOLS_VERSION = 2.5
+GRPC_VERSION = 1.4.2
HTTP_PARSER_VERSION = 2.6.2
LEVELDB_VERSION = 1.19
LIBEV_VERSION = 4.22
http://git-wip-us.apache.org/repos/asf/mesos/blob/90f7a7e1/configure.ac
----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index b6a6deb..ee3818d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -205,6 +205,11 @@ AC_ARG_ENABLE([debug],
option won't change them]),
[], [enable_debug=no])
+AC_ARG_ENABLE([grpc],
+ AS_HELP_STRING([--enable-grpc],
+ [enable grpc support]),
+ [], [enable_grpc=no])
+
AC_ARG_ENABLE([port-mapping-isolator],
AS_HELP_STRING([--enable-port-mapping-isolator],
[enable port mapping network isolator]),
@@ -331,6 +336,13 @@ AC_ARG_WITH([gmock],
location prefixed by the given path]),
[without_bundled_gmock=yes], [])
+AC_ARG_WITH([grpc],
+ AS_HELP_STRING([--with-grpc@<:@=DIR@:>@],
+ [excludes building and using the bundled grpc
+ package in lieu of an installed version at a
+ location prefixed by the given path]),
+ [without_bundled_grpc=yes], [])
+
AC_ARG_WITH([http-parser],
AS_HELP_STRING([--with-http-parser@<:@=DIR@:>@],
[excludes building and using the bundled http-parser
@@ -1463,8 +1475,11 @@ AM_CONDITIONAL([WITH_BUNDLED_NVML],
# CPPFLAGS while extending it by /include and to LDFLAGS while
# extending it by /lib.
if test -n "`echo $with_protobuf`"; then
- CPPFLAGS="$CPPFLAGS -isystem ${with_protobuf}/include"
- LDFLAGS="$LDFLAGS -L${with_protobuf}/lib"
+ PROTOBUF_CPPFLAGS="-isystem ${with_protobuf}/include"
+ PROTOBUF_LINKERFLAGS="-L${with_protobuf}/lib"
+
+ CPPFLAGS="$PROTOBUF_CPPFLAGS $CPPFLAGS"
+ LDFLAGS="$PROTOBUF_LINKERFLAGS $LDFLAGS"
fi
# Check if user has asked us to use a preinstalled protobuf, or if
@@ -1541,9 +1556,12 @@ fi
AM_CONDITIONAL([WITH_BUNDLED_PROTOBUF],
[test "x$with_bundled_protobuf" = "xyes"])
+AC_SUBST([PROTOBUF_CPPFLAGS])
AC_SUBST([PROTOBUF_JAR])
+AC_SUBST([PROTOBUF_LINKERFLAGS])
AC_SUBST([PROTOCOMPILER])
+
if test -n "`echo $with_libev`"; then
CPPFLAGS="$CPPFLAGS -isystem ${with_libev}/include"
LDFLAGS="$LDFLAGS -L${with_libev}/lib"
@@ -1728,8 +1746,11 @@ if test -z "`echo $with_ssl`" &&
fi
if test -n "`echo $with_ssl`"; then
- CPPFLAGS="-isystem ${with_ssl}/include $CPPFLAGS"
- LDFLAGS="-L${with_ssl}/lib $LDFLAGS"
+ SSL_CPPFLAGS="-isystem ${with_ssl}/include"
+ SSL_LINKERFLAGS="-L${with_ssl}/lib"
+
+ CPPFLAGS="$SSL_CPPFLAGS $CPPFLAGS"
+ LDFLAGS="$SSL_LINKERFLAGS $LDFLAGS"
fi
AC_CHECK_LIB([crypto], [RAND_poll], [found_crypto=yes])
@@ -1783,6 +1804,9 @@ fi
AM_CONDITIONAL([ENABLE_SSL], [test x"$enable_ssl" = "xyes"])
+AC_SUBST([SSL_CPPFLAGS])
+AC_SUBST([SSL_LINKERFLAGS])
+
# Check if user has asked us to use a preinstalled stout, or if
# they asked us to ignore all bundled libraries while compiling and
@@ -1930,6 +1954,97 @@ AC_SUBST([ZLIB_CPPFLAGS])
AC_SUBST([ZLIB_LINKERFLAGS])
+# Check if grpc prefix path was supplied and if so, add it to the
+# CPPFLAGS and LDFLAGS with respective /include and /lib path suffixes.
+if test -n "`echo $with_grpc`"; then
+ if test "x$without_bundled_protobuf" != "xyes"; then
+ AC_MSG_ERROR([protobuf not specified
+-------------------------------------------------------------------
+You have requested the use of a non-bundled grpc without specifying
+the protobuf library it uses. This is prohibited to avoid using
+conflicting versions of protobuf libraries. Please specify the
+location of protobuf used by grpc by providing a prefix path via
+--with-protobuf=DIR.
+-------------------------------------------------------------------
+ ])
+ fi
+
+ CPPFLAGS="$CPPFLAGS -isystem ${with_grpc}/include"
+ LDFLAGS="$LDFLAGS -L${with_grpc}/lib"
+fi
+
+if test "x$enable_grpc" = "xyes"; then
+ # Check if user has asked us to use a preinstalled grpc, or if they
+ # asked us to ignore all bundled libraries while compiling and linking.
+ if test "x$without_bundled_grpc" = "xyes" || \
+ test "x$enable_bundled" != "xyes"; then
+ # Use the system default prefix if not specified.
+ if test -n "`echo $with_grpc`"; then
+ GRPCPREFIX="${with_grpc}"
+ elif test "x$enable_bundled" != "xyes"; then
+ GRPCPREFIX="/usr"
+ fi
+
+ # Check if headers and library were located.
+ saved_LIBS="$LIBS"
+ LIBS="-lgrpc++ -lgrpc -lgpr $LIBS"
+ AC_LANG_PUSH([C++])
+ AC_CHECK_HEADERS([grpc++/grpc++.h],
+ [AC_LINK_IFELSE(
+ [AC_LANG_PROGRAM([#include <grpc++/grpc++.h>],
+ [grpc::Version();])],
+ [found_grpc=yes],
+ [found_grpc=no])],
+ [found_grpc=no])
+ AC_LANG_POP()
+ LIBS="$saved_LIBS"
+
+ if test "x$found_grpc" = "xyes"; then
+ with_bundled_grpc=no
+
+ AC_CHECK_TOOL([GRPC_CXX_PLUGIN_TEST],
+ [grpc_cpp_plugin],
+ [],
+ [$GRPCPREFIX/bin])
+
+ if test -z "`echo $GRPC_CXX_PLUGIN_TEST`"; then
+ AC_MSG_ERROR([grpc_cpp_plugin not found in PATH])
+ fi
+
+ GRPC_CXX_PLUGIN="$GRPCPREFIX/bin/grpc_cpp_plugin"
+
+ else
+ AC_MSG_ERROR([cannot find grpc
+-------------------------------------------------------------------
+You have requested the use of a non-bundled grpc but no suitable
+grpc could be found.
+
+You may want specify the location of grpc by providing a prefix
+path via --with-grpc=DIR, or check that the path you provided is
+correct if you are already doing this.
+-------------------------------------------------------------------
+ ])
+ fi
+ else
+ if test "x$found_ssl" != "xyes"; then
+ AC_MSG_ERROR([cannot find libssl
+-------------------------------------------------------------------
+libssl is required for gRPC to build.
+-------------------------------------------------------------------
+ ])
+ fi
+
+ with_bundled_grpc=yes
+ fi
+fi
+
+AM_CONDITIONAL([ENABLE_GRPC], [test "x$enable_grpc" = "xyes"])
+AM_CONDITIONAL([WITH_BUNDLED_GRPC],
+ [test "x$with_bundled_grpc" = "xyes"])
+
+AC_SUBST([GRPC_CXX_PLUGIN])
+
+
# Check if ZooKeeper prefix path was supplied and if so, add it to
# CPPFLAGS while extending it by /include/zookeeper and to LDFLAGS
# while extending it by /lib.
http://git-wip-us.apache.org/repos/asf/mesos/blob/90f7a7e1/src/Makefile.am
----------------------------------------------------------------------
diff --git a/src/Makefile.am b/src/Makefile.am
index 4cf3535..68fff14 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -39,6 +39,7 @@ GLOG = 3rdparty/glog-$(GLOG_VERSION)
GOOGLETEST = 3rdparty/googletest-release-$(GOOGLETEST_VERSION)
GMOCK = $(GOOGLETEST)/googlemock
GTEST = $(GOOGLETEST)/googletest
+GRPC = 3rdparty/grpc-$(GRPC_VERSION)
PROTOBUF = 3rdparty/protobuf-$(PROTOBUF_VERSION)
PICOJSON = 3rdparty/picojson-$(PICOJSON_VERSION)
NVML = 3rdparty/nvml-$(NVML_VERSION)
@@ -173,6 +174,19 @@ LIB_GLOG = -lglog
LDADD += -lglog
endif
+if ENABLE_GRPC
+if WITH_BUNDLED_GRPC
+LIB_GRPC = ../$(GRPC)/libs/opt/libgrpc++.a \
+ ../$(GRPC)/libs/opt/libgrpc.a \
+ ../$(GRPC)/libs/opt/libgpr.a \
+ -lssl -lcrypto
+GRPC_CPP_PLUGIN = ../$(GRPC)/bins/opt/grpc_cpp_plugin
+else
+LIB_GRPC = -lgrpc++ -lgrpc -lgpr -lssl -lcrypto
+GRPC_CPP_PLUGIN = @GRPC_CXX_PLUGIN@
+endif
+endif
+
if WITH_BUNDLED_LEVELDB
MESOS_CPPFLAGS += -I../$(LEVELDB)/include
@@ -1509,6 +1523,7 @@ libmesos_la_LIBTOOLFLAGS = --tag=CXX
libmesos_la_LIBADD = \
libmesos_no_3rdparty.la \
$(LIB_GLOG) \
+ $(LIB_GRPC) \
$(LIB_LEVELDB) \
$(LIB_PROCESS) \
$(LIB_PROTOBUF) \
[02/11] mesos git commit: Bundled gRPC v1.4.2 into 3rdparty libraries.
Posted by ji...@apache.org.
Bundled gRPC v1.4.2 into 3rdparty libraries.
The bundled package is generated with the following command:
git clone -b v1.4.2 https://github.com/grpc/grpc.git grpc-1.4.2
(cd grpc-1.4.2 && git submodule update --init third_party/cares)
tar zcvf grpc-1.4.2.tar.gz --exclude .git grpc-1.4.2
We download gRPC from GitHub instead of using the official tarball
because we need to use `git submodule` to download dependent third-party
packages for gRPC, and the necessary git metadata does not exist in the
official tarball.
gRPC also depends on openssl and protobuf 3. Since we have already
bundled the latter, the only system dependency is openssl.
Review: https://reviews.apache.org/r/61583
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/ab21cdf3
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/ab21cdf3
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/ab21cdf3
Branch: refs/heads/master
Commit: ab21cdf3432c64895862988d258a7527dcb6d196
Parents: 7a385a4
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Mon May 22 16:30:59 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:53:15 2017 -0700
----------------------------------------------------------------------
3rdparty/grpc-1.4.2.tar.gz | Bin 0 -> 6436282 bytes
1 file changed, 0 insertions(+), 0 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/ab21cdf3/3rdparty/grpc-1.4.2.tar.gz
----------------------------------------------------------------------
diff --git a/3rdparty/grpc-1.4.2.tar.gz b/3rdparty/grpc-1.4.2.tar.gz
new file mode 100644
index 0000000..f4dfe63
Binary files /dev/null and b/3rdparty/grpc-1.4.2.tar.gz differ
[09/11] mesos git commit: Building gRPC support in libprocess with
Autotools.
Posted by ji...@apache.org.
Building gRPC support in libprocess with Autotools.
This patch enables autotools to build gRPC support in libprocess along
with tests under Linux.
Review: https://reviews.apache.org/r/61160
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/e08593ae
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/e08593ae
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/e08593ae
Branch: refs/heads/master
Commit: e08593ae97c98f6b78405549a5cc7ad39233781d
Parents: 90f7a7e
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Mon Aug 7 17:24:06 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:06 2017 -0700
----------------------------------------------------------------------
3rdparty/libprocess/3rdparty/Makefile.am | 50 ++++++++++
3rdparty/libprocess/Makefile.am | 101 ++++++++++++++++++---
3rdparty/libprocess/configure.ac | 126 +++++++++++++++++++++++++-
3rdparty/libprocess/include/Makefile.am | 1 +
4 files changed, 259 insertions(+), 19 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/e08593ae/3rdparty/libprocess/3rdparty/Makefile.am
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/3rdparty/Makefile.am b/3rdparty/libprocess/3rdparty/Makefile.am
index 103a7cf..1070f6a 100644
--- a/3rdparty/libprocess/3rdparty/Makefile.am
+++ b/3rdparty/libprocess/3rdparty/Makefile.am
@@ -25,6 +25,8 @@ BUILT_SOURCES = # Initialized to enable using +=.
CONFIGURE_ARGS = @CONFIGURE_ARGS@ --enable-shared=no --with-pic --srcdir=.
# Set up building flags for non-bundled libraries.
+SSL_INCLUDE_FLAGS = @SSL_CPPFLAGS@
+SSL_LINKER_FLAGS = @SSL_LINKERFLAGS@
ZLIB_INCLUDE_FLAGS = @ZLIB_CPPFLAGS@
ZLIB_LINKER_FLAGS = @ZLIB_LINKERFLAGS@
@@ -39,6 +41,7 @@ GLOG = glog-$(GLOG_VERSION)
GOOGLETEST = googletest-release-$(GOOGLETEST_VERSION)
GMOCK = $(GOOGLETEST)/googlemock
GPERFTOOLS = gperftools-$(GPERFTOOLS_VERSION)
+GRPC = grpc-$(GRPC_VERSION)
GTEST = $(GOOGLETEST)/googletest
HTTP_PARSER = http-parser-$(HTTP_PARSER_VERSION)
LIBEV = libev-$(LIBEV_VERSION)
@@ -51,6 +54,7 @@ EXTRA_DIST = \
$(GLOG).tar.gz \
$(GOOGLETEST).tar.gz \
$(GPERFTOOLS).tar.gz \
+ $(GRPC).tar.gz \
$(HTTP_PARSER).tar.gz \
$(LIBEV).tar.gz \
$(PROTOBUF).tar.gz \
@@ -61,6 +65,10 @@ EXTRA_DIST = \
EXTRA_DIST += \
$(GLOG).patch
+# We patch grpc for CMake compilation issues.
+EXTRA_DIST += \
+ $(GRPC).patch
+
# We patch http-parser to avoid a name collision with our `flags` namespace.
EXTRA_DIST += \
$(HTTP_PARSER).patch
@@ -78,6 +86,7 @@ CLEAN_EXTRACTED = \
$(BOOST) \
$(CONCURRENTQUEUE) \
$(GLOG) \
+ $(GRPC) \
$(GOOGLETEST) \
$(GPERFTOOLS) \
$(HTTP_PARSER) \
@@ -185,6 +194,47 @@ $(PROTOBUF)-build-stamp: $(PROTOBUF)-stamp
touch $@
ALL_LOCAL += $(PROTOBUF)/src/libprotobuf.la
ALL_LOCAL += $(PROTOBUF)/src/protoc
+
+PROTOBUF_INCLUDE_FLAGS = -I$(abs_builddir)/$(PROTOBUF)/src
+PROTOBUF_LINKER_FLAGS = -L$(abs_builddir)/$(PROTOBUF)/src/.libs
+PROTOC = $(abs_builddir)/$(PROTOBUF)/src/protoc
+else
+$(PROTOBUF)-build-stamp:
+ touch $@
+
+PROTOBUF_INCLUDE_FLAGS = @PROTOBUF_CPPFLAGS@
+PROTOBUF_LINKER_FLAGS = @PROTOBUF_LINKERFLAGS@
+PROTOC = @PROTOCOMPILER@
+endif
+
+if ENABLE_GRPC
+if WITH_BUNDLED_GRPC
+LIB_GRPC = $(GRPC)/libs/opt/libgrpc++.a \
+ $(GRPC)/libs/opt/libgrpc.a \
+ $(GRPC)/libs/opt/libgpr.a
+
+$(LIB_GRPC): $(GRPC)-build-stamp
+
+$(GRPC)-build-stamp: $(GRPC)-stamp \
+ $(PROTOBUF)-build-stamp
+ cd $(GRPC) && \
+ CPPFLAGS="$(PROTOBUF_INCLUDE_FLAGS) \
+ $(SSL_INCLUDE_FLAGS) \
+ $(ZLIB_INCLUDE_FLAGS)" \
+ LDFLAGS="$(PROTOBUF_LINKER_FLAGS) \
+ $(SSL_LINKER_FLAGS) \
+ $(ZLIB_LINKERFLAGS)" \
+ $(MAKE) $(AM_MAKEFLAGS) \
+ HAS_PKG_CONFIG=false \
+ NO_PROTOC=false \
+ PROTOC=$(PROTOC)
+ touch $@
+
+ALL_LOCAL += $(LIB_GRPC)
+else
+$(GRPC)-build-stamp:
+ touch $@
+endif
endif
# Convenience library for gmock/gtest.
http://git-wip-us.apache.org/repos/asf/mesos/blob/e08593ae/3rdparty/libprocess/Makefile.am
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/Makefile.am b/3rdparty/libprocess/Makefile.am
index 90b7295..c2fa3f9 100644
--- a/3rdparty/libprocess/Makefile.am
+++ b/3rdparty/libprocess/Makefile.am
@@ -58,6 +58,7 @@ GLOG = $(BUNDLED_DIR)/glog-$(GLOG_VERSION)
GOOGLETEST = $(BUNDLED_DIR)/googletest-release-$(GOOGLETEST_VERSION)
GMOCK = $(GOOGLETEST)/googlemock
GPERFTOOLS = $(BUNDLED_DIR)/gperftools-$(GPERFTOOLS_VERSION)
+GRPC = $(BUNDLED_DIR)/grpc-$(GRPC_VERSION)
GTEST = $(GOOGLETEST)/googletest
HTTP_PARSER = $(BUNDLED_DIR)/http-parser-$(HTTP_PARSER_VERSION)
LIBEV = $(BUNDLED_DIR)/libev-$(LIBEV_VERSION)
@@ -111,6 +112,28 @@ $(LIB_GPERFTOOLS): $(GPERFTOOLS)-build-stamp
BUNDLED_DEPS += $(GPERFTOOLS)-build-stamp
endif
+if ENABLE_GRPC
+if WITH_BUNDLED_GRPC
+GRPC_INCLUDE_FLAGS = -I$(GRPC)/include
+LIB_GRPC = $(GRPC)/libs/opt/libgrpc++.a \
+ $(GRPC)/libs/opt/libgrpc.a \
+ $(GRPC)/libs/opt/libgpr.a \
+ -lssl -lcrypto
+GRPC_CPP_PLUGIN = $(GRPC)/bins/opt/grpc_cpp_plugin
+
+# NOTE: We need to specify the dependency between Protobuf and gRPC
+# here, otherwise `make` might launch concurrent gRPC and Protobuf
+# builds, which in turn trigger two racy Protobuf builds.
+$(GRPC)-build-stamp: $(PROTOBUF)-build-stamp
+
+$(LIB_GRPC): $(GRPC)-build-stamp
+BUNDLED_DEPS += $(GRPC)-build-stamp
+else
+LIB_GRPC = -lgrpc++ -lgrpc -lgpr -lssl -lcrypto
+GRPC_CPP_PLUGIN = @GRPC_CXX_PLUGIN@
+endif
+endif
+
if WITH_BUNDLED_HTTP_PARSER
HTTP_PARSER_INCLUDE_FLAGS = -I$(HTTP_PARSER)
LIB_HTTP_PARSER = $(BUNDLED_DIR)/libry_http_parser.la
@@ -140,7 +163,14 @@ BUNDLED_DEPS += $(PICOJSON)-stamp
endif
if WITH_BUNDLED_PROTOBUF
+PROTOBUF_INCLUDE_FLAGS = -I$(PROTOBUF)/src
+LIB_PROTOBUF = $(PROTOBUF)/src/libprotobuf.la
+PROTOC = $(PROTOBUF)/src/protoc
+$(LIB_PROTOBUF): $(PROTOBUF)-build-stamp
BUNDLED_DEPS += $(PROTOBUF)-build-stamp
+else
+LIB_PROTOBUF = -lprotobuf
+PROTOC = @PROTOCOMPILER@
endif
if WITH_BUNDLED_STOUT
@@ -192,13 +222,13 @@ libprocess_la_SOURCES = \
if ENABLE_SSL
libprocess_la_SOURCES += \
- src/jwt.cpp \
- src/jwt_authenticator.cpp \
- src/libevent_ssl_socket.cpp \
- src/libevent_ssl_socket.hpp \
- src/openssl.cpp \
- src/openssl.hpp \
- src/ssl/utilities.cpp
+ src/jwt.cpp \
+ src/jwt_authenticator.cpp \
+ src/libevent_ssl_socket.cpp \
+ src/libevent_ssl_socket.hpp \
+ src/openssl.cpp \
+ src/openssl.hpp \
+ src/ssl/utilities.cpp
endif
@@ -213,22 +243,29 @@ libprocess_la_CPPFLAGS = \
$(ELFIO_INCLUDE_FLAGS) \
$(GLOG_INCLUDE_FLAGS) \
$(GPERFTOOLS_INCLUDE_FLAGS) \
+ $(GRPC_INCLUDE_FLAGS) \
$(HTTP_PARSER_INCLUDE_FLAGS) \
$(LIB_EV_INCLUDE_FLAGS) \
$(PICOJSON_INCLUDE_FLAGS) \
+ $(PROTOBUF_INCLUDE_FLAGS) \
$(STOUT_INCLUDE_FLAGS) \
$(AM_CPPFLAGS)
+if ENABLE_GRPC
+libprocess_la_SOURCES += \
+ src/grpc.cpp
+endif
+
if ENABLE_LIBEVENT
libprocess_la_SOURCES += \
- src/libevent.hpp \
- src/libevent.cpp \
- src/libevent_poll.cpp
+ src/libevent.hpp \
+ src/libevent.cpp \
+ src/libevent_poll.cpp
else
- libprocess_la_SOURCES += \
- src/libev.hpp \
- src/libev.cpp \
- src/libev_poll.cpp
+libprocess_la_SOURCES += \
+ src/libev.hpp \
+ src/libev.cpp \
+ src/libev_poll.cpp
endif
if ENABLE_STATIC_LIBPROCESS
@@ -282,9 +319,35 @@ libprocess_tests_CPPFLAGS = \
$(libprocess_la_CPPFLAGS)
libprocess_tests_LDADD = \
+ $(LIB_GRPC) \
$(LIB_GMOCK) \
+ $(LIB_PROTOBUF) \
libprocess.la
+if ENABLE_GRPC
+libprocess_tests_SOURCES += \
+ src/tests/grpc_tests.cpp \
+ src/tests/grpc_tests.proto
+
+GRPC_TESTS_GENERATED_SOURCES = \
+ grpc_tests.pb.h \
+ grpc_tests.pb.cc \
+ grpc_tests.grpc.pb.h \
+ grpc_tests.grpc.pb.cc
+
+# Use a pseudo-target here to avoid multiple protoc invocations.
+$(GRPC_TESTS_GENERATED_SOURCES): grpc_tests_proto
+
+grpc_tests_proto: src/tests/grpc_tests.proto \
+ $(PROTOBUF)-build-stamp \
+ $(GRPC)-build-stamp
+ $(PROTOC) $< \
+ -I$(srcdir)/src/tests \
+ --cpp_out=$(builddir) \
+ --grpc_out=$(builddir) \
+ --plugin=protoc-gen-grpc=$(GRPC_CPP_PLUGIN)
+endif
+
if ENABLE_SSL
check_PROGRAMS += ssl-client
ssl_client_SOURCES = src/tests/ssl_client.cpp
@@ -333,7 +396,15 @@ libprocess_tests_LDADD += \
$(LIB_EV)
endif
-BUILT_SOURCES = $(BUNDLED_DEPS)
+BUILT_SOURCES = \
+ $(BUNDLED_DEPS) \
+ $(GRPC_TESTS_GENERATED_SOURCES)
+
+nodist_libprocess_tests_SOURCES = \
+ $(GRPC_TESTS_GENERATED_SOURCES)
+
+CLEANFILES = \
+ $(GRPC_TESTS_GENERATED_SOURCES)
# We use a check-local target for now to avoid the parallel test
# runner that ships with newer versions of autotools.
http://git-wip-us.apache.org/repos/asf/mesos/blob/e08593ae/3rdparty/libprocess/configure.ac
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/configure.ac b/3rdparty/libprocess/configure.ac
index 31556f7..1069d57 100644
--- a/3rdparty/libprocess/configure.ac
+++ b/3rdparty/libprocess/configure.ac
@@ -105,6 +105,11 @@ AC_ARG_ENABLE([debug],
option won't change them default: no]),
[], [enable_debug=no])
+AC_ARG_ENABLE([grpc],
+ AS_HELP_STRING([--enable-grpc],
+ [enable grpc support]),
+ [], [enable_grpc=no])
+
AC_ARG_ENABLE([install],
AS_HELP_STRING([--enable-install],
[install libprocess]),
@@ -188,6 +193,13 @@ AC_ARG_WITH([gmock],
location prefixed by the given path]),
[without_bundled_gmock=yes], [])
+AC_ARG_WITH([grpc],
+ AS_HELP_STRING([--with-grpc@<:@=DIR@:>@],
+ [excludes building and using the bundled grpc
+ package in lieu of an installed version at a
+ location prefixed by the given path]),
+ [without_bundled_grpc=yes], [])
+
AC_ARG_WITH([http-parser],
AS_HELP_STRING([--with-http-parser@<:@=DIR@:>@],
[excludes building and using the bundled http-parser
@@ -877,8 +889,11 @@ AM_CONDITIONAL([WITH_BUNDLED_PICOJSON], [test "x$with_bundled_picojson" = "xyes"
if test -n "`echo $with_protobuf`"; then
- CPPFLAGS="$CPPFLAGS -I${with_protobuf}/include"
- LDFLAGS="$LDFLAGS -L${with_protobuf}/lib"
+ PROTOBUF_CPPFLAGS="-isystem ${with_protobuf}/include"
+ PROTOBUF_LINKERFLAGS="-L${with_protobuf}/lib"
+
+ CPPFLAGS="$PROTOBUF_CPPFLAGS $CPPFLAGS"
+ LDFLAGS="$PROTOBUF_LINKERFLAGS $LDFLAGS"
fi
# Check if user has asked us to use a preinstalled protobuf, or if
@@ -917,6 +932,8 @@ if test "x$without_bundled_protobuf" = "xyes" || \
AC_MSG_ERROR([protoc not found in PATH])
fi
+ PROTOCOMPILER="$PROTOBUFPREFIX/bin/protoc"
+
else
AC_MSG_ERROR([cannot find protobuf
-------------------------------------------------------------------
@@ -936,6 +953,10 @@ fi
AM_CONDITIONAL([WITH_BUNDLED_PROTOBUF],
[test "x$with_bundled_protobuf" = "xyes"])
+AC_SUBST([PROTOBUF_CPPFLAGS])
+AC_SUBST([PROTOBUF_LINKERFLAGS])
+AC_SUBST([PROTOCOMPILER])
+
# Check if libssl prefix path was provided, and if so, add it to
# the CPPFLAGS and LDFLAGS with respective /include and /lib path
@@ -948,8 +969,11 @@ if test -z "`echo $with_ssl`" &&
fi
if test -n "`echo $with_ssl`"; then
- CPPFLAGS="-I${with_ssl}/include $CPPFLAGS"
- LDFLAGS="-L${with_ssl}/lib $LDFLAGS"
+ SSL_CPPFLAGS="-isystem ${with_ssl}/include"
+ SSL_LINKERFLAGS="-L${with_ssl}/lib"
+
+ CPPFLAGS="$SSL_CPPFLAGS $CPPFLAGS"
+ LDFLAGS="$SSL_LINKERFLAGS $LDFLAGS"
fi
AC_CHECK_LIB([crypto], [RAND_poll], [found_crypto=yes])
@@ -1003,6 +1027,9 @@ fi
AM_CONDITIONAL([ENABLE_SSL], [test x"$enable_ssl" = "xyes"])
+AC_SUBST([SSL_CPPFLAGS])
+AC_SUBST([SSL_LINKERFLAGS])
+
if test -n "`echo $with_stout`"; then
CPPFLAGS="$CPPFLAGS -I${with_stout}/include"
@@ -1108,6 +1135,97 @@ AC_SUBST([ZLIB_CPPFLAGS])
AC_SUBST([ZLIB_LINKERFLAGS])
+# Check if grpc prefix path was supplied and if so, add it to the
+# CPPFLAGS and LDFLAGS with respective /include and /lib path suffixes.
+if test -n "`echo $with_grpc`"; then
+ if test "x$without_bundled_protobuf" != "xyes"; then
+ AC_MSG_ERROR([protobuf not specified
+-------------------------------------------------------------------
+You have requested the use of a non-bundled grpc without specifying
+the protobuf library it uses. This is prohibited to avoid using
+conflicting versions of protobuf libraries. Please specify the
+location of protobuf used by grpc by providing a prefix path via
+--with-protobuf=DIR.
+-------------------------------------------------------------------
+ ])
+ fi
+
+ CPPFLAGS="$CPPFLAGS -isystem ${with_grpc}/include"
+ LDFLAGS="$LDFLAGS -L${with_grpc}/lib"
+fi
+
+if test "x$enable_grpc" = "xyes"; then
+ # Check if user has asked us to use a preinstalled grpc, or if they
+ # asked us to ignore all bundled libraries while compiling and linking.
+ if test "x$without_bundled_grpc" = "xyes" || \
+ test "x$enable_bundled" != "xyes"; then
+ # Use the system default prefix if not specified.
+ if test -n "`echo $with_grpc`"; then
+ GRPCPREFIX="${with_grpc}"
+ elif test "x$enable_bundled" != "xyes"; then
+ GRPCPREFIX="/usr"
+ fi
+
+ # Check if headers and library were located.
+ saved_LIBS="$LIBS"
+ LIBS="-lgrpc++ -lgrpc -lgpr $LIBS"
+ AC_LANG_PUSH([C++])
+ AC_CHECK_HEADERS([grpc++/grpc++.h],
+ [AC_LINK_IFELSE(
+ [AC_LANG_PROGRAM([#include <grpc++/grpc++.h>],
+ [grpc::Version();])],
+ [found_grpc=yes],
+ [found_grpc=no])],
+ [found_grpc=no])
+ AC_LANG_POP()
+ LIBS="$saved_LIBS"
+
+ if test "x$found_grpc" = "xyes"; then
+ with_bundled_grpc=no
+
+ AC_CHECK_TOOL([GRPC_CXX_PLUGIN_TEST],
+ [grpc_cpp_plugin],
+ [],
+ [$GRPCPREFIX/bin])
+
+ if test -z "`echo $GRPC_CXX_PLUGIN_TEST`"; then
+ AC_MSG_ERROR([grpc_cpp_plugin not found in PATH])
+ fi
+
+ GRPC_CXX_PLUGIN="$GRPCPREFIX/bin/grpc_cpp_plugin"
+
+ else
+ AC_MSG_ERROR([cannot find grpc
+-------------------------------------------------------------------
+You have requested the use of a non-bundled grpc but no suitable
+grpc could be found.
+
+You may want specify the location of grpc by providing a prefix
+path via --with-grpc=DIR, or check that the path you provided is
+correct if you are already doing this.
+-------------------------------------------------------------------
+ ])
+ fi
+ else
+ if test "x$found_ssl" != "xyes"; then
+ AC_MSG_ERROR([cannot find libssl
+-------------------------------------------------------------------
+libssl is required for gRPC to build.
+-------------------------------------------------------------------
+ ])
+ fi
+
+ with_bundled_grpc=yes
+ fi
+fi
+
+AM_CONDITIONAL([ENABLE_GRPC], [test "x$enable_grpc" = "xyes"])
+AM_CONDITIONAL([WITH_BUNDLED_GRPC],
+ [test "x$with_bundled_grpc" = "xyes"])
+
+AC_SUBST([GRPC_CXX_PLUGIN])
+
+
# Once all CXXFLAGS are completely assembled, make sure the we are not
# affected by libcxx undefined behavior,
# https://llvm.org/bugs/show_bug.cgi?id=28469.
http://git-wip-us.apache.org/repos/asf/mesos/blob/e08593ae/3rdparty/libprocess/include/Makefile.am
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/include/Makefile.am b/3rdparty/libprocess/include/Makefile.am
index 637654e..c5dc0bb 100644
--- a/3rdparty/libprocess/include/Makefile.am
+++ b/3rdparty/libprocess/include/Makefile.am
@@ -31,6 +31,7 @@ nobase_include_HEADERS = \
process/future.hpp \
process/gc.hpp \
process/gmock.hpp \
+ process/grpc.hpp \
process/gtest.hpp \
process/help.hpp \
process/http.hpp \
[05/11] mesos git commit: Added gRPC support in libprocess.
Posted by ji...@apache.org.
Added gRPC support in libprocess.
A gRPC client can create a `process::grpc::client::Runtime` instance and
use its `call()` method to send an asynchronous gRPC call to `Channel`
(representing a connection to a gRPC server), and get a future waiting
for the response. The `Runtime` class maintains a `CompletionQueue` to
manage all pending asynchronous gRPC calls, and spawns a thread waiting
for any response from the `CompletionQueue`. All gRPC calls using the
same `Runtime` copy would share the same `CompletionQueue` and thread.
Review: https://reviews.apache.org/r/61097
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/7ce40531
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/7ce40531
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/7ce40531
Branch: refs/heads/master
Commit: 7ce405318fad9284c67cf507480f8007f8219549
Parents: 70a4109
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Mon Jul 24 15:55:09 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:05 2017 -0700
----------------------------------------------------------------------
3rdparty/libprocess/include/process/grpc.hpp | 204 ++++++++++++++++++++++
3rdparty/libprocess/src/grpc.cpp | 96 ++++++++++
2 files changed, 300 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/7ce40531/3rdparty/libprocess/include/process/grpc.hpp
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/include/process/grpc.hpp b/3rdparty/libprocess/include/process/grpc.hpp
new file mode 100644
index 0000000..91c1cd9
--- /dev/null
+++ b/3rdparty/libprocess/include/process/grpc.hpp
@@ -0,0 +1,204 @@
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License
+
+#ifndef __PROCESS_GRPC_HPP__
+#define __PROCESS_GRPC_HPP__
+
+#include <atomic>
+#include <chrono>
+#include <memory>
+#include <thread>
+#include <type_traits>
+
+#include <google/protobuf/message.h>
+
+#include <grpc++/grpc++.h>
+
+#include <process/future.hpp>
+#include <process/owned.hpp>
+#include <process/process.hpp>
+
+#include <stout/duration.hpp>
+#include <stout/lambda.hpp>
+#include <stout/synchronized.hpp>
+#include <stout/try.hpp>
+
+
+// This file provides libprocess "support" for using gRPC. In
+// particular, it defines two wrapper classes: `Channel` (representing a
+// connection to a gRPC server) and `client::Runtime`, which integrates
+// an event loop waiting for gRPC responses, and provides the `call`
+// interface to create an asynchrous gRPC call and return a `Future`.
+
+
+#define GRPC_RPC(service, rpc) \
+ (&service::Stub::Async##rpc)
+
+namespace process {
+namespace grpc {
+
+// Forward declarations.
+namespace client { class Runtime; }
+
+
+/**
+ * A copyable interface to manage a connection to a gRPC server.
+ * All `Channel` copies share the same connection. Note that the
+ * connection is established lazily by the gRPC runtime library: the
+ * actual connection is delayed till an RPC call is made.
+ */
+class Channel
+{
+public:
+ Channel(const std::string& uri,
+ const std::shared_ptr<::grpc::ChannelCredentials>& credentials =
+ ::grpc::InsecureChannelCredentials())
+ : channel(::grpc::CreateChannel(uri, credentials)) {}
+
+private:
+ std::shared_ptr<::grpc::Channel> channel;
+
+ friend class client::Runtime;
+};
+
+
+namespace client {
+
+/**
+ * A copyable interface to manage an internal gRPC runtime instance for
+ * asynchronous gRPC calls. A gRPC runtime instance includes a gRPC
+ * `CompletionQueue` to manage outstanding requests, a looper thread to
+ * wait for any incoming responses from the `CompletionQueue`, and a
+ * process to handle the responses. All `Runtime` copies share the same
+ * gRPC runtime instance. Usually we only need a single gRPC runtime
+ * instance to handle all gRPC calls, but multiple instances can be
+ * instantiated for more parallelism or isolation.
+ * NOTE: The destruction of the internal gRPC runtime instance is a
+ * blocking operation: it waits for the managed process to terminate.
+ * The user should ensure that this only happens at shutdown.
+ */
+class Runtime
+{
+public:
+ Runtime() : data(new Data()) {}
+
+ /**
+ * Sends an asynchronous gRPC call.
+ *
+ * @param channel A connection to a gRPC server.
+ * @param rpc The asynchronous gRPC call to make. This can be obtained
+ * by the `GRPC_RPC(Service, RPC)` macro.
+ * @param request The request protobuf for the gRPC call.
+ * @return a `Future` waiting for a response protobuf.
+ */
+ template <typename Stub, typename Request, typename Response>
+ Future<Response> call(
+ const Channel& channel,
+ std::unique_ptr<::grpc::ClientAsyncResponseReader<Response>>(Stub::*rpc)(
+ ::grpc::ClientContext*,
+ const Request&,
+ ::grpc::CompletionQueue*),
+ const Request& request)
+ {
+ static_assert(
+ std::is_convertible<Request*, google::protobuf::Message*>::value,
+ "Request must be a protobuf message");
+
+ synchronized (data->lock) {
+ if (data->terminating) {
+ return Failure("Runtime has been terminated.");
+ }
+
+ std::shared_ptr<::grpc::ClientContext> context(
+ new ::grpc::ClientContext());
+
+ // TODO(chhsiao): Allow the caller to specify a timeout.
+ context->set_deadline(
+ std::chrono::system_clock::now() + std::chrono::seconds(5));
+
+ // Create a `Promise` and a callback lambda as a tag and invokes
+ // an asynchronous gRPC call through the `CompletionQueue`
+ // managed by `data`. The `Promise` will be set by the callback
+ // upon server response.
+ std::shared_ptr<Promise<Response>> promise(new Promise<Response>);
+ promise->future().onDiscard([=] { context->TryCancel(); });
+
+ std::shared_ptr<Response> response(new Response());
+ std::shared_ptr<::grpc::Status> status(new ::grpc::Status());
+
+ std::shared_ptr<::grpc::ClientAsyncResponseReader<Response>> reader(
+ (Stub(channel.channel).*rpc)(context.get(), request, &data->queue));
+
+ reader->Finish(
+ response.get(),
+ status.get(),
+ new lambda::function<void()>(
+ // NOTE: `context` and `reader` need to be held on in
+ // order to get updates for the ongoing RPC, and thus
+ // are captured here. The lambda itself will later be
+ // retrieved and managed in `Data::loop()`.
+ [context, reader, response, status, promise]() {
+ CHECK(promise->future().isPending());
+ if (promise->future().hasDiscard()) {
+ promise->discard();
+ } else if (status->ok()) {
+ promise->set(*response);
+ } else {
+ promise->fail(status->error_message());
+ }
+ }));
+
+ return promise->future();
+ }
+ }
+
+ /**
+ * Asks the internal gRPC runtime instance to shut down the
+ * `CompletionQueue`, which would stop its looper thread, drain and
+ * fail all pending gRPC calls in the `CompletionQueue`, then
+ * asynchronously join the looper thread.
+ */
+ void terminate();
+
+ /**
+ * @return A `Future` waiting for all pending gRPC calls in the
+ * `CompletionQueue` of the internal gRPC runtime instance to be
+ * drained and the looper thread to be joined.
+ */
+ Future<Nothing> wait();
+
+private:
+ struct Data
+ {
+ Data();
+ ~Data();
+
+ void loop();
+ void terminate();
+
+ std::unique_ptr<std::thread> looper;
+ ::grpc::CompletionQueue queue;
+ ProcessBase process;
+ std::atomic_flag lock;
+ bool terminating;
+ Promise<Nothing> terminated;
+ };
+
+ std::shared_ptr<Data> data;
+};
+
+} // namespace client {
+
+} // namespace grpc {
+} // namespace process {
+
+#endif // __PROCESS_GRPC_HPP__
http://git-wip-us.apache.org/repos/asf/mesos/blob/7ce40531/3rdparty/libprocess/src/grpc.cpp
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/src/grpc.cpp b/3rdparty/libprocess/src/grpc.cpp
new file mode 100644
index 0000000..3ba5bc5
--- /dev/null
+++ b/3rdparty/libprocess/src/grpc.cpp
@@ -0,0 +1,96 @@
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License
+
+#include <process/dispatch.hpp>
+#include <process/grpc.hpp>
+#include <process/id.hpp>
+
+namespace process {
+namespace grpc {
+
+namespace client {
+
+void Runtime::terminate()
+{
+ data->terminate();
+}
+
+
+Future<Nothing> Runtime::wait()
+{
+ return data->terminated.future();
+}
+
+
+Runtime::Data::Data()
+ : process(ID::generate("__grpc_client__")),
+ lock(ATOMIC_FLAG_INIT),
+ terminating(false)
+{
+ spawn(process);
+
+ // The looper thread can only be created here since it need to happen
+ // after `queue` is initialized.
+ looper.reset(new std::thread(&Runtime::Data::loop, this));
+}
+
+
+Runtime::Data::~Data()
+{
+ terminate();
+ process::wait(process);
+}
+
+
+void Runtime::Data::loop()
+{
+ void* tag;
+ bool ok;
+
+ while (queue.Next(&tag, &ok)) {
+ // The returned callback object is managed by the `callback` shared
+ // pointer, so if we get a regular event from the `CompletionQueue`,
+ // then the object would be captured by the following lambda
+ // dispatched to `process`; otherwise it would be reclaimed here.
+ std::shared_ptr<lambda::function<void()>> callback(
+ reinterpret_cast<lambda::function<void()>*>(tag));
+ if (ok) {
+ dispatch(process, [=] { (*callback)(); });
+ }
+ }
+
+ dispatch(process, [this] {
+ // NOTE: This is a blocking call. However, the thread is guaranteed
+ // to be exiting, therefore the amount of blocking time should be
+ // short (just like other syscalls we invoke).
+ looper->join();
+ // Terminate `process` after all events are drained.
+ process::terminate(process, false);
+ terminated.set(Nothing());
+ });
+}
+
+
+void Runtime::Data::terminate()
+{
+ synchronized (lock) {
+ if (!terminating) {
+ terminating = true;
+ queue.Shutdown();
+ }
+ }
+}
+
+} // namespace client {
+
+} // namespace grpc {
+} // namespace process {
[11/11] mesos git commit: Refactored OpenSSL library checks in Mesos.
Posted by ji...@apache.org.
Refactored OpenSSL library checks in Mesos.
Since gRPC requires OpenSSL, we checks if libssl and libcrypto exist no
matter if the `--enable-ssl` flag is on. This enables us to have a
non-SSL-enabled Mesos build with gRPC support. Also fixed errors that
a bad linker might link the configure tests with libssl unnecessarily,
which would cause runtime failures if libssl is not in the runtime
library search path.
Review: https://reviews.apache.org/r/61433
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/fa8b3111
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/fa8b3111
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/fa8b3111
Branch: refs/heads/master
Commit: fa8b311152cd5780db2fb962cb88e46333764991
Parents: f2b16ca
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Tue Aug 8 18:04:28 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:55:06 2017 -0700
----------------------------------------------------------------------
configure.ac | 39 +++++++++++++++++++++++++--------------
1 file changed, 25 insertions(+), 14 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/fa8b3111/configure.ac
----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index 08f4339..b6a6deb 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1700,7 +1700,7 @@ __EOF__
# distcheck failure on OSX by leaking build artefacts (.dsym).
AS_IF([test "x${ac_cv_env_CFLAGS_set}" = "x"],
[SASL_TEST_CFLAGS=""], [SASL_TEST_CFLAGS=$CFLAGS])
-$CC crammd5_installed.c $CPPFLAGS $SASL_TEST_CFLAGS $LDFLAGS -lsasl2 $LIBS \
+$CC crammd5_installed.c $CPPFLAGS $SASL_TEST_CFLAGS $LDFLAGS -lsasl2 \
-o crammd5_installed 2>&1 >/dev/null
# Run the test binary and get its output.
@@ -1732,27 +1732,33 @@ if test -n "`echo $with_ssl`"; then
LDFLAGS="-L${with_ssl}/lib $LDFLAGS"
fi
+AC_CHECK_LIB([crypto], [RAND_poll], [found_crypto=yes])
+
+AC_CHECK_HEADERS([openssl/ssl.h],
+ [AC_CHECK_LIB([ssl],
+ [SSL_CTX_new],
+ [found_ssl=yes],
+ [],
+ [-lcrypto])])
+
if test "x$enable_ssl" = "xyes"; then
- AC_CHECK_HEADERS([openssl/ssl.h],
- [AC_CHECK_LIB([ssl],
- [SSL_CTX_new],
- [],
- [AC_MSG_ERROR([cannot find libssl
--------------------------------------------------------------------
-libssl is required for an SSL-enabled build.
--------------------------------------------------------------------
- ])])],
- [AC_MSG_ERROR([cannot find libssl headers
+ if test "x$found_ssl" != "xyes"; then
+ AC_MSG_ERROR([cannot find libssl
-------------------------------------------------------------------
libssl is required for an SSL-enabled build.
-------------------------------------------------------------------
- ])])
+ ])
+ fi
- AC_CHECK_LIB([crypto], [RAND_poll], [], [AC_MSG_ERROR([cannot find libcrypto
+ if test "x$found_crypto" != "xyes"; then
+ AC_MSG_ERROR([cannot find libcrypto
-------------------------------------------------------------------
libcrypto is required for an SSL-enabled build.
-------------------------------------------------------------------
- ])])
+ ])
+ fi
+
+ LIBS="-lssl -lcrypto $LIBS"
if test "x$enable_libevent" = "xyes"; then
AC_CHECK_HEADERS([event2/bufferevent_ssl.h],
@@ -1975,6 +1981,10 @@ AM_CONDITIONAL([WITH_BUNDLED_ZOOKEEPER],
# affected by libcxx undefined behavior,
# https://llvm.org/bugs/show_bug.cgi?id=28469.
AC_MSG_CHECKING([C++ standard library for undefined behaviour with selected optimization level])
+# NOTE: We clear `LIBS` here to prevent linking in libraries unrelated
+# to the test. These libraries might not be in the linker lookup paths.
+saved_LIBS="$LIBS"
+LIBS=""
AC_LANG_PUSH([C++])
AC_RUN_IFELSE([
AC_LANG_SOURCE([[
@@ -2000,6 +2010,7 @@ AC_RUN_IFELSE([
]
)
AC_LANG_POP([C++])
+LIBS="$saved_LIBS"
# NOTE: Do not update any compiler or linker settings (e.g. CXXFLAGS,
[03/11] mesos git commit: Removed `--disable-zlib` and fixed
`--with-zlib` for libprocess.
Posted by ji...@apache.org.
Removed `--disable-zlib` and fixed `--with-zlib` for libprocess.
Added `--with-zlib` for specifying a custom zlib path. For third-party
libraries that does not support `--with-zlib=DIR`, we introduce new
variables `ZLIB_CPPFLAGS` and `ZLIB_LINKERFLAGS` so that they can be
used to set up `CPPFLAGS` and `LDFLAGS` when building those libraries.
Review: https://reviews.apache.org/r/61509
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/7a385a46
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/7a385a46
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/7a385a46
Branch: refs/heads/master
Commit: 7a385a464fe1b76bbd7b3009d8f043fbe0eff6f9
Parents: 30914ea
Author: Chun-Hung Hsiao <ch...@mesosphere.io>
Authored: Tue Aug 8 13:44:11 2017 -0700
Committer: Jie Yu <yu...@gmail.com>
Committed: Thu Aug 10 16:53:15 2017 -0700
----------------------------------------------------------------------
3rdparty/libprocess/3rdparty/Makefile.am | 15 +++++++--
3rdparty/libprocess/configure.ac | 44 ++++++++++++++++++---------
2 files changed, 42 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/7a385a46/3rdparty/libprocess/3rdparty/Makefile.am
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/3rdparty/Makefile.am b/3rdparty/libprocess/3rdparty/Makefile.am
index d05b6e6..103a7cf 100644
--- a/3rdparty/libprocess/3rdparty/Makefile.am
+++ b/3rdparty/libprocess/3rdparty/Makefile.am
@@ -24,6 +24,10 @@ BUILT_SOURCES = # Initialized to enable using +=.
# '--srcdir=...' when configuring.
CONFIGURE_ARGS = @CONFIGURE_ARGS@ --enable-shared=no --with-pic --srcdir=.
+# Set up building flags for non-bundled libraries.
+ZLIB_INCLUDE_FLAGS = @ZLIB_CPPFLAGS@
+ZLIB_LINKER_FLAGS = @ZLIB_LINKERFLAGS@
+
# Directory which holds bundled dependencies
BUNDLED_DIR = $(top_srcdir)/..
@@ -168,9 +172,16 @@ endif
if WITH_BUNDLED_PROTOBUF
$(PROTOBUF)/src/protoc $(PROTOBUF)/src/libprotobuf.la: $(PROTOBUF)-build-stamp
+# NOTE: The `--with-zlib` flag works differently between libprocess and
+# Protobuf. Protobuf uses `--with-zlib` as an on/off switch for data
+# compression instead of a path specifier for ZLib, so we need to set up
+# the `CPPFLAGS` and `LDFLAGS` if `--with-zlib=DIR` is given.
$(PROTOBUF)-build-stamp: $(PROTOBUF)-stamp
- cd $(PROTOBUF) && ./configure $(CONFIGURE_ARGS) && \
- $(MAKE) $(AM_MAKEFLAGS)
+ cd $(PROTOBUF) && \
+ ./configure $(CONFIGURE_ARGS) \
+ CPPFLAGS="$(ZLIB_INCLUDE_FLAGS)" \
+ LDFLAGS="$(ZLIB_LINKER_FLAGS)" && \
+ $(MAKE) $(AM_MAKEFLAGS)
touch $@
ALL_LOCAL += $(PROTOBUF)/src/libprotobuf.la
ALL_LOCAL += $(PROTOBUF)/src/protoc
http://git-wip-us.apache.org/repos/asf/mesos/blob/7a385a46/3rdparty/libprocess/configure.ac
----------------------------------------------------------------------
diff --git a/3rdparty/libprocess/configure.ac b/3rdparty/libprocess/configure.ac
index 29b69b9..db45de0 100644
--- a/3rdparty/libprocess/configure.ac
+++ b/3rdparty/libprocess/configure.ac
@@ -138,12 +138,6 @@ AC_ARG_ENABLE([static-unimplemented],
functions default: no]),
[], [enable_static_unimplemented=no])
-AC_ARG_ENABLE([zlib],
- AS_HELP_STRING([--disable-zlib],
- [disables zlib compression, which means the webui
- will be far less responsive; not recommended]),
- [], [enable_zlib=yes])
-
###############################################################################
# Optional packages.
@@ -242,6 +236,11 @@ AC_ARG_WITH([svn],
[specify where to locate the svn-1 library]),
[], [])
+AC_ARG_WITH([zlib],
+ AS_HELP_STRING([--with-zlib=@<:@=DIR@:>@],
+ [specify where to locate the zlib library]),
+ [], [])
+
###############################################################################
# Miscellaneous flags/library/tool checks.
@@ -1075,17 +1074,32 @@ libsubversion-1 is required for libprocess to build.
])])
-# Check if we should/can build with libz.
-if test "x$enable_zlib" = "xyes"; then
- AC_CHECK_LIB([z], [deflate, gzread, gzwrite, inflate], [],
- [AC_MSG_ERROR([cannot find libz
+# Check if zlib prefix path was provided, and if so, add it to
+# the CPPFLAGS and LDFLAGS with respective /include and /lib path
+# suffixes.
+if test -n "`echo $with_zlib`" ; then
+ ZLIB_CPPFLAGS="-isystem ${with_zlib}/include"
+ ZLIB_LINKERFLAGS="-L${with_zlib}/lib"
+
+ CPPFLAGS="$ZLIB_CPPFLAGS $CPPFLAGS"
+ LDFLAGS="$ZLIB_LINKERFLAGS $LDFLAGS"
+fi
+
+AC_CHECK_HEADERS([zlib.h],
+ [AC_CHECK_LIB([z], [deflate, gzread, gzwrite, inflate], [],
+ [AC_MSG_ERROR([cannot find libz
-------------------------------------------------------------------
-This means HTTP responses will be slower because we cannot use
-compression; you probably want to download and install zlib, but
-you can get away without it by doing --disable-zlib.
+libz is required for libprocess to build.
-------------------------------------------------------------------
- ])])
-fi
+ ])])],
+ [AC_MSG_ERROR([cannot find libz headers
+-------------------------------------------------------------------
+libz headers are required for libprocess to build.
+-------------------------------------------------------------------
+ ])])
+
+AC_SUBST([ZLIB_CPPFLAGS])
+AC_SUBST([ZLIB_LINKERFLAGS])
# Once all CXXFLAGS are completely assembled, make sure the we are not