You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by kirill-morozov-epam <gi...@git.apache.org> on 2016/08/27 15:04:55 UTC
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
GitHub user kirill-morozov-epam opened a pull request:
https://github.com/apache/bigtop/pull/143
BIGTOP-2524: Add Greenplum component
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kirill-morozov-epam/bigtop bigtop_gpdb
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bigtop/pull/143.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #143
----
commit 2225845fdbed804a3b930ca0d9c9365aaaecee0d
Author: osboxes.org <os...@mn.moscow.epam.com>
Date: 2016-08-19T06:56:21Z
gpdb(GreenPlum) component added
commit 8f642c62a5d3d2622ae3b5409f0cde4739029983
Author: osboxes.org <os...@mn.moscow.epam.com>
Date: 2016-08-19T13:24:04Z
gpdb parameters in hiera; hadoop manifest fixed for puppet parser=future
commit ddd7e39187a42ffa94d2c7dc21595bc689e23095
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-22T11:34:02Z
add start server in test manifest
commit 71f1fe7d351fc8d3753c3f719841dd6ca8f7f568
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-22T13:19:27Z
fix flink puppet manifest according paresr=future
commit 244d9700a2a8cf22fed25d1bdc1228ca4c58bbb7
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-22T15:58:07Z
fix hbase and hadoop-zookeper puppet manifests according parser=future
commit 185dab2d8997a7d65ba54d8d6769066929bbfb0c
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T08:00:22Z
fix hadoop-oozie puppet manifests according parser=future
commit 51b19b6137848f26d84b9649975b1f78c7cb7607
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T08:00:53Z
fix hadoop-pig puppet manifests according parser=future
commit 15987415e2cd8371a8bd8dbfda43c814608f3f26
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T08:03:30Z
fix qfs puppet manifest according parser=future
commit 4ccec9a411db994f0f4a6f8ee3114fefcb868bcb
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T08:44:51Z
fix spark puppet manifest according parser=future
commit 0e2bedeb17ca14cb703767dc417bc3a89bf9ac6c
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T14:13:30Z
remove duplicate package in ubuntu
commit 297ff3b92ddbff99b28b70dee2c41bdc356ae36b
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T14:32:42Z
full path to gpdb installation, instead enviroment var
commit 11f315a8d4f6ad3f918b7023009d22e895478fb7
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T14:34:45Z
add Package resources for ubuntu
commit 53b8043a76292e789860d3d8e0655a1ab465fb3f
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-23T16:16:01Z
add require to install_packages for prepare_db_dirs; fixed libffi-dev package for ubuntu
commit 1c79c4d21a3e6eb60641c0669640b8ee5968dc93
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-24T09:49:45Z
fixed require for prepare_db_dirs; fixed require libffi-dev for ubuntu
commit 84b053e29d4215e7ed2a5e14a5c02da588dc1cf6
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-25T12:35:00Z
libapr1 compability gpdb build on openSuse issue
commit 1d0c5b28eac45b766d27c45d37b2c1312f8618b7
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-26T07:30:39Z
hack for openSUSE autorequire disabled
commit b2d2875670ee1978df227a14cd5d7ff0142c9119
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-26T07:56:55Z
"mode" in "file" resource fixed according parser=future
commit af239887b4f81026cad4adcea7d826222741600b
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-26T10:01:51Z
BIGTOP-2521: code fixed according require parser=future
commit 4d58b41b08d2537f757cc3519478bb347680dc73
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-26T11:49:57Z
del uneeded comments; clean code
commit c66b97fa5ea215b81e93cb7cc2963f1d962fea85
Author: Kirill Morozov <ki...@epam.ru>
Date: 2016-08-26T12:01:23Z
exclude gpdb script templates from rat report
commit 3735c8ebd7aabc6ee0d34e06a0b261f32c615dc2
Author: Artur Papikyan <ar...@epam.com>
Date: 2016-08-26T13:12:42Z
BIGTOP-2523. in order to install puppet 3.8 we need to get it from puppet repo
commit 3e7331bc9553faa4964e529e8541e08c29178155
Author: Kirill Morozov <ki...@epam.com>
Date: 2016-08-27T06:31:39Z
Merge branch 'BIGTOP-2523' into bigtop_gpdb
commit 9c92dfa57447f5b6f25c0050911869feb43927f6
Author: Kirill Morozov <ki...@epam.com>
Date: 2016-08-27T06:35:32Z
Del commented code
commit 6f3ae313830cbf134bc4ba7adad6684233b0dbc8
Author: Kirill Morozov <ki...@epam.com>
Date: 2016-08-27T06:36:20Z
Merge branch 'BIGTOP-2521' into bigtop_gpdb
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77867595
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
--- End diff --
An overall comment about the puppet code: it is one of the best practices to align "=>" to the same position within the structure. This way it is easier to make changes/read the code. Could you please reformat it accordingly. Sorry for not mentioning this earlier.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77867183
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
--- End diff --
Perhaps I am missing something, but it seems this class serves no real purpose. Could you please explain the need for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77923188
--- Diff: bigtop-packages/src/deb/gpdb/control ---
@@ -0,0 +1,24 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+Source: gpdb
+Section: misc
+Priority: extra
+Maintainer: Bigtop <de...@bigtop.apache.org>
+Standards-Version: 4.3.99.0
+Homepage: https://github.com/greenplum-db/gpdb
+
+Package: gpdb
+Architecture: all
+Description: GreenPlum
--- End diff --
Let's make it a bit more descriptive, like
`Greenplum MPP database enginer`
or something like that
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77866807
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
--- End diff --
Minor: should be saying "stop_if_running"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77923498
--- Diff: bigtop-packages/src/rpm/gpdb/SPECS/gpdb.spec ---
@@ -0,0 +1,65 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+%define man_dir %{_mandir}
+
+%if %{?suse_version:1}0
+%define bin_gpdb /usr/lib/gpdb
+%define doc_gpdb %{_docdir}/%{name}
+%define autorequire no
+%else
+%define bin_gpdb /usr/lib/gpdb
+%define doc_gpdb %{_docdir}/%{name}-%{gpdb_version}
+%define autorequire yes
+%endif
+%define debug_package %{nil}
+
+Name: gpdb
+Version: %{gpdb_version}
+Release: %{gpdb_release}
+Summary: GreenPlum
+URL: https://github.com/greenplum-db/gpdb
+Group: Development/Libraries
+Buildroot: %{_topdir}/INSTALL/%{name}-%{version}
+License: ASL 2.0
+Source0: gpdb-4.3.99.0.tar.gz
+Source1: do-component-build
+Source2: install_gpdb.sh
+Source3: do-component-configure
+AutoReqProv: %{autorequire}
+
+%description
+gpdb
+
+%prep
+%autosetup
+
+%build
+bash %{SOURCE3} %{bin_gpdb}
+bash %{SOURCE1}
+
+%install
+%__rm -rf $RPM_BUILD_ROOT
+bash %{SOURCE2}
+mkdir -p $RPM_BUILD_ROOT%{bin_gpdb}
+cp -f -r %{bin_gpdb}/* $RPM_BUILD_ROOT/%{bin_gpdb}
+%__rm -rf %{bin_gpdb}
+
+%files
+%defattr(-,root,root)
+%{bin_gpdb}
+
--- End diff --
2 extra blanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by kirill-morozov-epam <gi...@git.apache.org>.
Github user kirill-morozov-epam commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r79561910
--- Diff: bigtop-deploy/puppet/modules/gpdb/templates/init-db.sh ---
@@ -0,0 +1,4 @@
+#!/bin/sh
+if [ ! -d $1 ]; then
+export LD_LIBRARY_PATH=<%= gp_home%>/lib:/lib;<%= gp_home%>/bin/initdb -E UNICODE -D $1 --locale=en_US.utf8 --max_connections=750 --shared_buffers=128000kB --is_filerep_mirrored=no --backend_output=$1.initdb
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77866848
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
--- End diff --
extra blank line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on the issue:
https://github.com/apache/bigtop/pull/143
Well, looks like I can not build the RPM while running build as a non-privileged user (which is what we do always). I am getting this:
```
make[1]: Entering directory `/ws/build/gpdb/rpm/BUILD/gpdb-4.3.99.0/src'
/bin/mkdir -p '/usr/lib/gpdb/lib/postgresql/pgxs/src'
/bin/mkdir: cannot create directory '/usr/lib/gpdb': Permission denied
make[1]: *** [installdirs-local] Error 1
make[1]: Leaving directory `/ws/build/gpdb/rpm/BUILD/gpdb-4.3.99.0/src'
make: *** [install] Error 2
```
Looks like a prefix is missed somewhere or something.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by kirill-morozov-epam <gi...@git.apache.org>.
Github user kirill-morozov-epam commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r79561897
--- Diff: bigtop-deploy/puppet/modules/gpdb/README.md ---
@@ -0,0 +1,19 @@
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+# Package instalation notes
+
+Greenplum scripts need the next python libs: psutil, paramiko.
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r79451565
--- Diff: bigtop-deploy/puppet/modules/gpdb/README.md ---
@@ -0,0 +1,19 @@
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+# Package instalation notes
+
+Greenplum scripts need the next python libs: psutil, paramiko.
--- End diff --
`the next python` should read `the following python`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #143: BIGTOP-2524: Add Greenplum component
Posted by kirill-morozov-epam <gi...@git.apache.org>.
Github user kirill-morozov-epam commented on the issue:
https://github.com/apache/bigtop/pull/143
Issue build package under non-privileged user fixed in https://github.com/apache/bigtop/pull/143/commits/a19feb2f560feaa96432d26fd5c6a668f1b8e5b1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r79451840
--- Diff: bigtop-deploy/puppet/modules/gpdb/templates/init-db.sh ---
@@ -0,0 +1,4 @@
+#!/bin/sh
+if [ ! -d $1 ]; then
+export LD_LIBRARY_PATH=<%= gp_home%>/lib:/lib;<%= gp_home%>/bin/initdb -E UNICODE -D $1 --locale=en_US.utf8 --max_connections=750 --shared_buffers=128000kB --is_filerep_mirrored=no --backend_output=$1.initdb
--- End diff --
2-spaces indentation is missing in this line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77866981
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
+ class { 'gpdb::common::configure_master_node':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs']
+ }
+
+ class { 'gpdb::common::stop_master_in_admin_mode':
+ base_dir => "$db_base_dir",
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ require => Class['gpdb::common::configure_master_node']
+ }
+
+ gpdb::server{"start":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::stop_master_in_admin_mode'],
+ start_or_stop => running,
+ }
+
--- End diff --
2 extra blanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #143: BIGTOP-2524: Add Greenplum component
Posted by kirill-morozov-epam <gi...@git.apache.org>.
Github user kirill-morozov-epam commented on the issue:
https://github.com/apache/bigtop/pull/143
This is issue in gpdb build scripts. Fix it in https://github.com/greenplum-db/gpdb/pull/1146
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77868168
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
+ class { 'gpdb::common::configure_master_node':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs']
+ }
+
+ class { 'gpdb::common::stop_master_in_admin_mode':
+ base_dir => "$db_base_dir",
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ require => Class['gpdb::common::configure_master_node']
+ }
+
+ gpdb::server{"start":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::stop_master_in_admin_mode'],
+ start_or_stop => running,
+ }
+
+
+
+ class gpadmin_user{
+ group { 'gpadmin':
+ ensure => 'present',
+ }
+
+ user { 'gpadmin':
+ ensure => 'present',
+ system => false,
+ managehome => true,
+ groups => 'gpadmin',
+ shell => '/bin/bash',
+ }
+ }
+
+ class install_packages{
--- End diff --
Also, I suggest we add these packages to the Bigtop's deploy docker image in a separate JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77923713
--- Diff: build.gradle ---
@@ -129,7 +129,8 @@ rat {
"bigtop-deploy/puppet/README.md",
"bigtop-deploy/puppet/modules/qfs/README.md",
"bigtop-deploy/puppet/config/site.csv.example",
- "dl/**"
+ "dl/**",
+ "bigtop-deploy/puppet/modules/gpdb/templates/**"
--- End diff --
please also add this exclude to the pom.xml file - we still use the old maven based build for some release tasks, but slowly getting rid of it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77867992
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
+ class { 'gpdb::common::configure_master_node':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs']
+ }
+
+ class { 'gpdb::common::stop_master_in_admin_mode':
+ base_dir => "$db_base_dir",
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ require => Class['gpdb::common::configure_master_node']
+ }
+
+ gpdb::server{"start":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::stop_master_in_admin_mode'],
+ start_or_stop => running,
+ }
+
+
+
+ class gpadmin_user{
+ group { 'gpadmin':
+ ensure => 'present',
+ }
+
+ user { 'gpadmin':
+ ensure => 'present',
+ system => false,
+ managehome => true,
+ groups => 'gpadmin',
+ shell => '/bin/bash',
+ }
+ }
+
+ class install_packages{
--- End diff --
Hmm, is there any reason the package dependencies aren't declared in the packages themselves?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77868402
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
+ class { 'gpdb::common::configure_master_node':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs']
+ }
+
+ class { 'gpdb::common::stop_master_in_admin_mode':
+ base_dir => "$db_base_dir",
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ require => Class['gpdb::common::configure_master_node']
+ }
+
+ gpdb::server{"start":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::stop_master_in_admin_mode'],
+ start_or_stop => running,
+ }
+
+
+
+ class gpadmin_user{
+ group { 'gpadmin':
+ ensure => 'present',
+ }
+
+ user { 'gpadmin':
+ ensure => 'present',
+ system => false,
+ managehome => true,
+ groups => 'gpadmin',
+ shell => '/bin/bash',
+ }
+ }
+
+ class install_packages{
+ case $operatingsystem{
+ /(?i:(centos|fedora))/: {
+ notice('rpm')
+ yumrepo { "epel":
+ baseurl => "http://download.fedoraproject.org/pub/epel/7/\$basearch",
+ descr => "epel packages",
+ enabled => 1,
+ gpgcheck => 0,
+ }
+
+ package { ["libffi-devel"]:
+ ensure => latest,
+ }
+
+ package { ["python-lockfile"]:
+ ensure => latest,
+ }
+
+ package { ["psutil"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["paramiko"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["python-pip"]:
+ ensure => latest,
+ require => [
+ Yumrepo["epel"],
+ Package["libffi-devel"],
+ Package["python-lockfile"],
+ ],
+ }
+
+ }
+ /(?i:(SLES|opensuse))/: {
+ notice('suse')
+ }
+ Amazon: { notice('amazon') }
+ /(Ubuntu|Debian)/: {
+ notice('ubuntu')
+ package { ["libffi-dev"]:
+ ensure => latest,
+ }
+
+ package { ["python-lockfile"]:
+ ensure => latest,
+ }
+
+ package { ["psutil"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["paramiko"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["python-pip"]:
+ ensure => latest,
+ require => [
+ Package["libffi-dev"],
+ Package["python-lockfile"],
+ ],
+ }
+ }
+ }
+
+ package { ["gpdb"]:
+ ensure => latest,
+ }
+ }
+
+
+
+ class prepare_file_structure($base_dir = undef){
+
+ file { "$base_dir":
+ ensure => 'directory',
+ owner => 'gpadmin',
+ group => 'gpadmin',
+ mode => '0770',
+ }
+
+ file { "$base_dir/master":
+ ensure => 'directory',
+ owner => 'gpadmin',
+ group => 'gpadmin',
+ mode => '0770',
+ }
+
+ file { "$base_dir/primary":
+ ensure => 'directory',
+ owner => 'gpadmin',
+ group => 'gpadmin',
+ mode => '0770',
+ }
+ }
+
+ class prepare_db_dirs($nodes = undef, $base_dir = undef, $gp_home = undef, $master_port = undef, $segment_port_prefix = undef){
+ file { '/home/gpadmin/init-db.sh':
+ content => template('gpdb/init-db.sh'),
+ require => [
+ File["$base_dir/primary"],
+ File["$base_dir/master"]],
+ owner => 'gpadmin',
+ group => 'gpadmin',
+ mode => '0700',
+ }
+ file { '/home/gpadmin/stop-db.sh':
+ content => template('gpdb/stop-db.sh'),
+ # require => [Exec["create_master_db$base_dir/master/gpseg-1"]],
+ owner => 'gpadmin',
+ group => 'gpadmin',
+ mode => '0700',
+ }
+
+ each($nodes) |$index, $value| {
+ notice($value)
+ if ($::fqdn == $value) {
+ $dbid = $index+1
+ $content = $index-1
+ $hostname = $value
+ $segment_nodes_count=size($nodes)-1
+ if ($dbid == 1){
+ $port = $master_port
+ $db_sub_dir = "master"
+ $m_options = "master"
+ $x_options = "-x 0 -E"
+ $z_options = "$segment_nodes_count"
+ } else{
+ $port = "$segment_port_prefix${content}"
+ $db_sub_dir = "primary"
+ $m_options = "mirrorless"
+ $x_options = ""
+ $z_options = "$segment_nodes_count"
+ }
+ $db_dir = "$base_dir/${db_sub_dir}/gpseg${content}"
+ # notice("${dbid} - ${content} - ${port} - ${hostname} - ${db_dir}")
--- End diff --
I guess this comment doesn't have to be here anymore.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r79450456
--- Diff: bigtop-packages/src/rpm/gpdb/SPECS/gpdb.spec ---
@@ -0,0 +1,63 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+%define man_dir %{_mandir}
+
+%if %{?suse_version:1}0
+%define bin_gpdb /usr/lib/gpdb
+%define doc_gpdb %{_docdir}/%{name}
+%define autorequire no
+%else
+%define bin_gpdb /usr/lib/gpdb
+%define doc_gpdb %{_docdir}/%{name}-%{gpdb_version}
+%define autorequire yes
+%endif
+%define debug_package %{nil}
+
+Name: gpdb
+Version: %{gpdb_version}
+Release: %{gpdb_release}
+Summary: Greenplum MPP database enginer
+URL: https://github.com/greenplum-db/gpdb
+Group: Development/Libraries
+Buildroot: %{_topdir}/INSTALL/%{name}-%{version}
+License: ASL 2.0
+Source0: gpdb-4.3.99.0.tar.gz
--- End diff --
Can we use already defined `%{gpdb_version}` instead of the hard-coded version string?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77868216
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
+ class { 'gpdb::common::configure_master_node':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs']
+ }
+
+ class { 'gpdb::common::stop_master_in_admin_mode':
+ base_dir => "$db_base_dir",
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ require => Class['gpdb::common::configure_master_node']
+ }
+
+ gpdb::server{"start":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::stop_master_in_admin_mode'],
+ start_or_stop => running,
+ }
+
+
+
+ class gpadmin_user{
+ group { 'gpadmin':
+ ensure => 'present',
+ }
+
+ user { 'gpadmin':
+ ensure => 'present',
+ system => false,
+ managehome => true,
+ groups => 'gpadmin',
+ shell => '/bin/bash',
+ }
+ }
+
+ class install_packages{
+ case $operatingsystem{
+ /(?i:(centos|fedora))/: {
+ notice('rpm')
+ yumrepo { "epel":
+ baseurl => "http://download.fedoraproject.org/pub/epel/7/\$basearch",
+ descr => "epel packages",
+ enabled => 1,
+ gpgcheck => 0,
+ }
+
+ package { ["libffi-devel"]:
+ ensure => latest,
+ }
+
+ package { ["python-lockfile"]:
+ ensure => latest,
+ }
+
+ package { ["psutil"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["paramiko"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["python-pip"]:
+ ensure => latest,
+ require => [
+ Yumrepo["epel"],
+ Package["libffi-devel"],
+ Package["python-lockfile"],
+ ],
+ }
+
+ }
+ /(?i:(SLES|opensuse))/: {
+ notice('suse')
+ }
+ Amazon: { notice('amazon') }
+ /(Ubuntu|Debian)/: {
+ notice('ubuntu')
+ package { ["libffi-dev"]:
+ ensure => latest,
+ }
+
+ package { ["python-lockfile"]:
+ ensure => latest,
+ }
+
+ package { ["psutil"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["paramiko"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["python-pip"]:
+ ensure => latest,
+ require => [
+ Package["libffi-dev"],
+ Package["python-lockfile"],
+ ],
+ }
+ }
+ }
+
+ package { ["gpdb"]:
+ ensure => latest,
+ }
+ }
+
--- End diff --
2 extra blanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77866936
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
--- End diff --
Extra blank line: inconsistent with the rest of the code
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #143: BIGTOP-2524: Add Greenplum component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/143#discussion_r77868322
--- Diff: bigtop-deploy/puppet/modules/gpdb/manifests/init.pp ---
@@ -0,0 +1,440 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+class gpdb {
+
+
+ class deploy($roles) {
+ if ("gpdb-master" in $roles or "gpdb-segment" in $roles) {
+ include gpdb::common
+ }
+ }
+
+
+ class common($nodes, $gp_home, $db_base_dir, $master_db_port, $segment_db_port_prefix) {
+
+ include stdlib
+
+ class { 'gpdb::common::install_packages': }
+
+ class { 'gpdb::common::gpadmin_user':}
+
+ class { 'gpdb::common::prepare_file_structure':
+ base_dir => "$db_base_dir",
+ require => Class['gpdb::common::gpadmin_user']
+ }
+
+ class { 'gpdb::common::prepare_db_dirs':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => [Class['gpdb::common::prepare_file_structure'], Package["gpdb"]]
+ }
+
+ gpdb::server{"stop_if_runned":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs'],
+ start_or_stop => stopped,
+ }
+
+
+ class { 'gpdb::common::configure_master_node':
+ base_dir => "$db_base_dir",
+ nodes => $nodes,
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ segment_port_prefix => $segment_db_port_prefix,
+ require => Class['gpdb::common::prepare_db_dirs']
+ }
+
+ class { 'gpdb::common::stop_master_in_admin_mode':
+ base_dir => "$db_base_dir",
+ gp_home => $gp_home,
+ master_port => $master_db_port,
+ require => Class['gpdb::common::configure_master_node']
+ }
+
+ gpdb::server{"start":
+ nodes => $gpdb::common::nodes,
+ gp_home => $gpdb::common::gp_home,
+ db_base_dir => $gpdb::common::db_base_dir,
+ master_db_port => $gpdb::common::master_db_port,
+ segment_db_port_prefix => $gpdb::common::segment_db_port_prefix,
+ require => Class['gpdb::common::stop_master_in_admin_mode'],
+ start_or_stop => running,
+ }
+
+
+
+ class gpadmin_user{
+ group { 'gpadmin':
+ ensure => 'present',
+ }
+
+ user { 'gpadmin':
+ ensure => 'present',
+ system => false,
+ managehome => true,
+ groups => 'gpadmin',
+ shell => '/bin/bash',
+ }
+ }
+
+ class install_packages{
+ case $operatingsystem{
+ /(?i:(centos|fedora))/: {
+ notice('rpm')
+ yumrepo { "epel":
+ baseurl => "http://download.fedoraproject.org/pub/epel/7/\$basearch",
+ descr => "epel packages",
+ enabled => 1,
+ gpgcheck => 0,
+ }
+
+ package { ["libffi-devel"]:
+ ensure => latest,
+ }
+
+ package { ["python-lockfile"]:
+ ensure => latest,
+ }
+
+ package { ["psutil"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["paramiko"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["python-pip"]:
+ ensure => latest,
+ require => [
+ Yumrepo["epel"],
+ Package["libffi-devel"],
+ Package["python-lockfile"],
+ ],
+ }
+
+ }
+ /(?i:(SLES|opensuse))/: {
+ notice('suse')
+ }
+ Amazon: { notice('amazon') }
+ /(Ubuntu|Debian)/: {
+ notice('ubuntu')
+ package { ["libffi-dev"]:
+ ensure => latest,
+ }
+
+ package { ["python-lockfile"]:
+ ensure => latest,
+ }
+
+ package { ["psutil"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["paramiko"]:
+ ensure => latest,
+ provider => pip,
+ require => Package["python-pip"],
+ }
+
+ package { ["python-pip"]:
+ ensure => latest,
+ require => [
+ Package["libffi-dev"],
+ Package["python-lockfile"],
+ ],
+ }
+ }
+ }
+
+ package { ["gpdb"]:
+ ensure => latest,
+ }
+ }
+
+
+
+ class prepare_file_structure($base_dir = undef){
+
+ file { "$base_dir":
+ ensure => 'directory',
+ owner => 'gpadmin',
+ group => 'gpadmin',
+ mode => '0770',
--- End diff --
is the mode 770 correct? Not 755 or 750?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---