You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by ad...@apache.org on 2019/06/04 22:37:45 UTC
[kudu] branch master updated (c45002d -> ee2898e)
This is an automated email from the ASF dual-hosted git repository.
adar pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from c45002d [rebalance] remove unused function declaration and fix typo error in comment.
new fa17619 KUDU-2398: Kudu kubernetes statefulset manifest
new ee2898e thirdparty: fix memkind build race
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
kubernetes/kudu-statefulset.yaml | 212 +++++++++++++++++++++++++++++++++++++++
thirdparty/build-definitions.sh | 3 +-
2 files changed, 214 insertions(+), 1 deletion(-)
create mode 100644 kubernetes/kudu-statefulset.yaml
[kudu] 01/02: KUDU-2398: Kudu kubernetes statefulset manifest
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit fa176193795041d04839386d5bd8a92eb0a08041
Author: SandishKumarHN <sa...@gmail.com>
AuthorDate: Wed May 22 16:55:29 2019 -0700
KUDU-2398: Kudu kubernetes statefulset manifest
it builds kubernetes statefulset kudu cluster, statefulset manages the
deployment and scaling of kudu server pods.
creating kubernetes kudu cluster: kubectl create -f kudu-statefulset.yaml
to destroy: kubectl delete -f kudu-statefulset.yaml
scale: kubectl scale statefulset kudu-tserver --replicas=5
Change-Id: I3f87a70b3c6ac87bc531306855c80011dce3a61d
Reviewed-on: http://gerrit.cloudera.org:8080/13407
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
---
kubernetes/kudu-statefulset.yaml | 212 +++++++++++++++++++++++++++++++++++++++
1 file changed, 212 insertions(+)
diff --git a/kubernetes/kudu-statefulset.yaml b/kubernetes/kudu-statefulset.yaml
new file mode 100644
index 0000000..39c7143
--- /dev/null
+++ b/kubernetes/kudu-statefulset.yaml
@@ -0,0 +1,212 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# More about kubernetes statefulset https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
+# build kudu docker image from ../docker folder: ../docker/docker-build.sh
+# create statefulset kudu kubernetes cluster : kubectl create -f kudu-statefulset.yaml
+# get kudu master url: minikube service kudu-master-ui --url
+# scale kudu tserver cluster : kubectl scale statefulset kudu-tserver --replicas=5
+# destroy kudu kubernetes cluster: kubectl delete -f kudu-statefulset.yaml
+
+apiVersion: v1
+kind: Service
+metadata:
+ name: kudu-masters
+ labels:
+ app: kudu-master
+spec:
+ clusterIP: None
+ ports:
+ - name: ui
+ port: 8051
+ - name: rpc-port
+ port: 8050
+ selector:
+ app: kudu-master
+---
+# headless service for masters
+apiVersion: v1
+kind: Service
+metadata:
+ name: kudu-master-ui
+ labels:
+ app: kudu-master
+spec:
+ clusterIP:
+ ports:
+ - name: ui
+ port: 8051
+ selector:
+ app: kudu-master
+ type: NodePort
+ target-port:
+ externalTrafficPolicy: Local
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: kudu-master
+ labels:
+ app: kudu-master
+spec:
+ serviceName: kudu-masters
+ podManagementPolicy: "Parallel"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: kudu-master
+ template:
+ metadata:
+ labels:
+ app: kudu-master
+ spec:
+ containers:
+ - name: kudu-master
+ image: apache/kudu
+ imagePullPolicy: IfNotPresent
+ env:
+ - name: GET_HOSTS_FROM
+ value: dns
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ - name: POD_NAME
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.name
+ - name: KUDU_MASTERS
+ value: "kudu-master-0.kudu-masters.default.svc.cluster.local,kudu-master-1.kudu-masters.default.svc.cluster.local,kudu-master-2.kudu-masters.default.svc.cluster.local"
+ args: ["master"]
+ ports:
+ - containerPort: 8050
+ name: master-ui
+ - containerPort: 8051
+ name: master-rpc
+ volumeMounts:
+ - name: datadir
+ mountPath: /mnt/data0
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - kudu-master
+ topologyKey: kubernetes.io/hostname
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes: [ "ReadWriteOnce" ]
+ resources:
+ requests:
+ storage: 10Gi
+ storageClassName: standard
+
+---
+# headless service for tservers
+apiVersion: v1
+kind: Service
+metadata:
+ name: kudu-tservers
+ namespace: default
+ labels:
+ app: kudu-tserver
+spec:
+ clusterIP: None
+ ports:
+ - name: ui
+ port: 7050
+ - name: rpc-port
+ port: 7051
+ selector:
+ app: kudu-tserver
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: kudu-tserver
+ labels:
+ app: kudu-tserver
+spec:
+ serviceName: kudu-tservers
+ podManagementPolicy: "Parallel"
+ replicas: 3
+ selector:
+ matchLabels:
+ app: kudu-tserver
+ template:
+ metadata:
+ labels:
+ app: kudu-tserver
+ spec:
+ containers:
+ - name: kudu-tserver
+ image: apache/kudu
+ imagePullPolicy: IfNotPresent
+ env:
+ - name: GET_HOSTS_FROM
+ value: dns
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ - name: POD_NAME
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.name
+ - name: KUDU_MASTERS
+ value: "kudu-master-0.kudu-masters.default.svc.cluster.local,kudu-master-1.kudu-masters.default.svc.cluster.local,kudu-master-2.kudu-masters.default.svc.cluster.local"
+ args: ["tserver"]
+ ports:
+ - containerPort: 7050
+ name: tserver-ui
+ - containerPort: 7051
+ name: tserver-rpc
+ volumeMounts:
+ - name: datadir
+ mountPath: /mnt/data0
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - kudu-tserver
+ topologyKey: kubernetes.io/hostname
+ updateStrategy:
+ type: RollingUpdate
+ volumeClaimTemplates:
+ - metadata:
+ name: datadir
+ spec:
+ accessModes: [ "ReadWriteOnce" ]
+ resources:
+ requests:
+ storage: 10Gi
+ storageClassName: standard
[kudu] 02/02: thirdparty: fix memkind build race
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit ee2898e18ae87625e2e6180210348ea069de8c5e
Author: Adar Dembo <ad...@cloudera.com>
AuthorDate: Tue Jun 4 14:26:46 2019 -0700
thirdparty: fix memkind build race
On rare occasions, the libmemkind.a produced by the memkind build was
missing its jemalloc symbols. This then led to Kudu linkage failures if Kudu
was using static linkage (i.e. in RELEASE builds).
This turned out to be due to a build race of our own making: we can't ask
make to run the install-exec and static_lib targets simultaneously. Both
targets depend on libmemkind.a, but static_lib performs some important
post-processing on libmemkind.a that must complete before install-exec
copies libmemkind.a into the installed directory. And by offering both
targets on the command line, make can run them in parallel once their common
dependency (libmemkind.a) is built.
Change-Id: I1242e733e7d282d1e7471656ea81349be427aa1f
Reviewed-on: http://gerrit.cloudera.org:8080/13513
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Kudu Jenkins
---
thirdparty/build-definitions.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index 4df64f6..406603b 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -805,7 +805,8 @@ build_memkind() {
# Building unit tests and memkind examples in memkind with clang will prevent some
# header files from being included which will lead to failures. So we do a minimal
# installation here.
- make -j$PARALLEL $EXTRA_MAKEFLAGS static_lib install-exec install-data
+ make -j$PARALLEL $EXTRA_MAKEFLAGS static_lib
+ make -j$PARALLEL $EXTRA_MAKEFLAGS install-exec install-data
# install jemalloc header files
cd $MEMKIND_BDIR/jemalloc/obj && make -j$PARALLEL $EXTRA_MAKEFLAGS install_include