You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2020/07/29 07:17:34 UTC
[GitHub] [bigtop] guyuqi opened a new pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
guyuqi opened a new pull request #657:
URL: https://github.com/apache/bigtop/pull/657
Implement build scripts for Kibana binaries and package scripts/resources
to create corresponding deb package.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-679891997
Thanks for revising the PR and elaboration @guyuqi! I thought I deployed and started elasticsearch too, but I'll check again and try to follow the steps you explained.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn merged pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
sekikn merged pull request #657:
URL: https://github.com/apache/bigtop/pull/657
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi commented on a change in pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
guyuqi commented on a change in pull request #657:
URL: https://github.com/apache/bigtop/pull/657#discussion_r464061497
##########
File path: bigtop-packages/src/common/kibana/do-component-build
##########
@@ -0,0 +1,41 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -ex
+
+. `dirname ${0}`/bigtop.bom
+
+
+# Install the specific version of nodejs defined in Kibana
+curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
+export NVM_DIR="$HOME/.nvm"
+[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
+[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
+
+# Build issues: shasum command not found in Centos/Fedora
+if [ -f /etc/os-release ]; then
+ . /etc/os-release
+fi
+case ${ID} in
+ centos | fedora)
+ sudo ln -s /usr/bin/sha1sum /usr/bin/shasum
+ ;;
+esac
Review comment:
Yep，I added `perl-Digest-SHA` in` bigtop_toolchain/manifests/packages.pp ` to fix issues and add `ln -s /usr/bin/sha1sum /usr/bin/shasum` in `do-component-build` for debugging but forgot to remove it.
Fixed it in the new commit, thanks.
##########
File path: bigtop-packages/src/common/kibana/install_kibana.sh
##########
@@ -0,0 +1,118 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -e
+
+usage() {
+ echo "
+usage: $0 <options>
+ Required not-so-options:
+ --distro-dir=DIR path to distro specific files (debian/RPM)
+ --build-dir=DIR path to dist dir
+ --prefix=PREFIX path to install into
+
+ Optional options:
+ --doc-dir=DIR path to install docs into [/usr/share/doc/elasticsearch]
+ --lib-dir=DIR path to install bits [/usr/lib/elasticsearch]
+ --installed-lib-dir=DIR path where lib-dir will end up on target system
+ --bin-dir=DIR path to install bins [/usr/bin]
+ --examples-dir=DIR path to install examples [doc-dir/examples]
+ ... [ see source for more similar options ]
+ "
+ exit 1
+}
+
+OPTS=$(getopt \
+ -n $0 \
+ -o '' \
+ -l 'prefix:' \
+ -l 'distro-dir:' \
+ -l 'doc-dir:' \
+ -l 'lib-dir:' \
+ -l 'installed-lib-dir:' \
+ -l 'bin-dir:' \
+ -l 'initd-dir:' \
+ -l 'build-dir:' -- "$@")
+
+if [ $? != 0 ] ; then
+ usage
+fi
+
+eval set -- "$OPTS"
+while true ; do
+ case "$1" in
+ --prefix)
+ PREFIX=$2 ; shift 2
+ ;;
+ --distro-dir)
+ DISTRO_DIR=$2 ; shift 2
+ ;;
+ --build-dir)
+ BUILD_DIR=$2 ; shift 2
+ ;;
+ --doc-dir)
+ DOC_DIR=$2 ; shift 2
+ ;;
+ --lib-dir)
+ LIB_DIR=$2 ; shift 2
+ ;;
+ --)
+ shift; break
+ ;;
+ *)
+ echo "Unknown option: $1"
+ usage
+ exit 1
+ ;;
+ esac
+done
+
+for var in PREFIX BUILD_DIR DISTRO_DIR ; do
+ if [ -z "$(eval "echo \$$var")" ]; then
+ echo Missing param: $var
+ usage
+ fi
+done
+
+MAN_DIR=${MAN_DIR:-/usr/share/man/man1}
+DOC_DIR=${DOC_DIR:-/usr/share/doc/kibana}
+LIB_DIR=${LIB_DIR:-/usr/lib/kibana}
+BIN_DIR=${BIN_DIR:-/usr/bin}
+ETC_DIR=${ETC_DIR:-/etc/kibana}
+CONF_DIR=${CONF_DIR:-${ETC_DIR}/conf.dist}
+INITD_DIR=${INITD_DIR:-/etc/init.d}
+
+VAR_DIR=$PREFIX/var
+
+install -d -m 0755 $PREFIX/$MAN_DIR
+install -d -m 0755 $PREFIX/$DOC_DIR
+install -d -m 0755 $PREFIX/$LIB_DIR
+install -d -m 0755 $PREFIX/$ETC_DIR
+install -d -m 0755 $PREFIX/$CONF_DIR
+install -d -m 0755 $PREFIX/$BIN_DIR
+
+# Extract files and copy to LIB_DIR
+tar zxf $BUILD_DIR/kibana-*-linux-*64.tar.gz -C $PREFIX/$LIB_DIR --strip-components 1
Review comment:
Fixed. thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
guyuqi commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-674806129
Removed unused sysV-init file and disable sysV kibana service.
kibana depends on Elasticsearch. If elasticsearch doesn't start, kibaba would fail to start.
So users would start kibana manually by `$KIBANA_HOME//bin/start-kibana` after they deployed their elasticsearch cluster.
The kibana deploy configuration : `bigtop-deploy/puppet/modules/kibana/`
And this PR leverages it to delpoy a container to run the smoke tests as a separate uses case.
>>On Debian 9 and 10, I failed to install the package with the following error. I tried to add write permission to /usr/lib/kibana/optimize recursively, but the preceding error was not resolved.
I'm sorry I could not reproduce it.
I installed the Deb package on Debian-9 : `dpkg -i kibana_5.4.1-1_all.deb`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi edited a comment on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
guyuqi edited a comment on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-667661543
Kibana are successfully built on CentOS 7/8 ,Fedora 31, ubuntu16/18 and debian9/10.
The smoke tests are passed in Fedora31, ubuntu18.04 and Centos7/8.
Elasticsearch should also be deployed before kibana smoke tests:
```
components: [elasticsearch, kibana]
enable_local_repo: true
smoke_test_components: [kibana]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-667958263
Thanks for the fix @guyuqi, I encountered other problems with the revised PR...
* On CentOS 7 and 8, the kibana package doesn't seem to install its init script.
```
$ sudo yum install kibana
(snip)
==============================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================
Installing:
kibana x86_64 5.4.1-1.el7 Bigtop_0 40 M
(snip)
Complete!
$ sudo service kibana status
Redirecting to /bin/systemctl status kibana.service
Unit kibana.service could not be found.
$ rpm -qlp output/kibana/x86_64/kibana-5.4.1-1.el7.x86_64.rpm | grep /etc
/etc/kibana/conf.dist
/etc/kibana/conf.dist/conf
/etc/kibana/conf.dist/kibana.yml
/usr/lib/kibana/node/lib/node_modules/npm/node_modules/request/node_modules/http-signature/node_modules/sshpk/node_modules/dashdash/etc
/usr/lib/kibana/node/lib/node_modules/npm/node_modules/request/node_modules/http-signature/node_modules/sshpk/node_modules/dashdash/etc/dashdash.bash_completion.in
/usr/lib/kibana/node_modules/flot-charts/examples/axes-time-zones/tz/etcetera
```
* On Debian 9 and 10, I failed to install the package with the following error. I tried to add write permission to /usr/lib/kibana/optimize recursively, but the preceding error was not resolved.
```
Aug 03 10:26:19 debian10.localdomain kibana[6554]: Starting kibana ...
Aug 03 10:26:19 debian10.localdomain kibana[6554]: Usage: serve [options]
Aug 03 10:26:19 debian10.localdomain kibana[6554]: Run the kibana server
Aug 03 10:26:19 debian10.localdomain kibana[6554]: Options:
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -h, --help output usage information
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -e, --elasticsearch <uri> Elasticsearch instance
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -c, --config <path> Path to the config file, can be changed with the CONFIG_PATH environment variable as well. Use multiple --config args to include multiple config files.
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -p, --port <port> The port to bind to
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -q, --quiet Prevent all logging except errors
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -Q, --silent Prevent all logging
Aug 03 10:26:19 debian10.localdomain kibana[6554]: --verbose Turns on verbose logging
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -H, --host <host> The host to bind to
Aug 03 10:26:19 debian10.localdomain kibana[6554]: -l, --log-file <path> The file to log to
Aug 03 10:26:19 debian10.localdomain kibana[6554]: --plugin-dir <path> A path to scan for plugins, this can be specified multiple times to specify multiple directories
Aug 03 10:26:19 debian10.localdomain kibana[6554]: --plugin-path <path> A path to a plugin which should be included by the server, this can be specified multiple times to specify multiple paths
Aug 03 10:26:19 debian10.localdomain kibana[6554]: --plugins <path> an alias for --plugin-dir
Aug 03 10:26:19 debian10.localdomain kibana[6554]: fs.js:641
Aug 03 10:26:19 debian10.localdomain kibana[6554]: return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
Aug 03 10:26:19 debian10.localdomain kibana[6554]: ^
Aug 03 10:26:19 debian10.localdomain kibana[6554]: Error: EACCES: permission denied, open '/usr/lib/kibana/optimize/.babelcache.json'
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at Error (native)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at Object.fs.openSync (fs.js:641:18)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at Object.fs.writeFileSync (fs.js:1347:33)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at process.save (/usr/lib/kibana/node_modules/babel-register/lib/cache.js:45:16)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at emitOne (events.js:96:13)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at process.emit (events.js:188:7)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at process.exit (internal/process.js:164:15)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at outputHelpIfNecessary (/usr/lib/kibana/node_modules/commander/index.js:1072:15)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at Command.listener (/usr/lib/kibana/node_modules/commander/index.js:268:5)
Aug 03 10:26:19 debian10.localdomain kibana[6554]: at emitTwo (events.js:106:13)
Aug 03 10:26:19 debian10.localdomain runuser[6604]: pam_unix(runuser:session): session closed for user kibana
Aug 03 10:26:19 debian10.localdomain kibana[6554]: .
Aug 03 10:26:19 debian10.localdomain systemd[1]: kibana.service: Control process exited, code=exited, status=1/FAILURE
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
guyuqi commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-667661543
The smoke test:
```
components: [elasticsearch, kibana]
enable_local_repo: true
smoke_test_components: [kibana]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-679545388
Thanks for the fix @guyuqi, I confirmed that building kibana and installing its deb succeeded on Debian 9/10 and Ubuntu 16.04/18.04 with the latest PR.
I have two additional comments:
* It seems that the `Source3: kibana.default` line in kibana.spec should be removed.
* I also tried to run the smoke test on Debian and Ubuntu, but on those distros, start-kibana in TestKibanaSmoke.groovy didn't seem to return and the test didn't finish. Is it the same for you, or it normally finishes on your environment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-680733632
Referring to your instruction, I confirmed the smoke test succeeded on Debian 9 and Ubuntu 16.04/18.04. Thanks @guyuqi!
(Maybe I mistakenly set "http://localhost" (not "http://localhost:9200") to `$ELASTICSEARCH_URL` when I ran the smoke test inside docker container before)
+1, will merge into master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on a change in pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
sekikn commented on a change in pull request #657:
URL: https://github.com/apache/bigtop/pull/657#discussion_r461646595
##########
File path: bigtop-packages/src/common/kibana/do-component-build
##########
@@ -0,0 +1,41 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -ex
+
+. `dirname ${0}`/bigtop.bom
+
+
+# Install the specific version of nodejs defined in Kibana
+curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
+export NVM_DIR="$HOME/.nvm"
+[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
+[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
+
+# Build issues: shasum command not found in Centos/Fedora
+if [ -f /etc/os-release ]; then
+ . /etc/os-release
+fi
+case ${ID} in
+ centos | fedora)
+ sudo ln -s /usr/bin/sha1sum /usr/bin/shasum
+ ;;
+esac
Review comment:
I tried to run `./gradlew kibana-pkg` after `./gradlew toolchain` on CentOS 7/8 and Fedora 31 with this PR, and came across the following error.
```
+ sudo ln -s /usr/bin/sha1sum /usr/bin/shasum
ln: failed to create symbolic link '/usr/bin/shasum': File exists
```
/usr/bin/shasum seems to be installed via perl-Digest-SHA, so these lines look like unnecessary, at least on x86_64 platform?
```
$ rpm -qf /usr/bin/shasum
perl-Digest-SHA-6.02-440.fc31.x86_64
```
##########
File path: bigtop-packages/src/common/kibana/install_kibana.sh
##########
@@ -0,0 +1,118 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -e
+
+usage() {
+ echo "
+usage: $0 <options>
+ Required not-so-options:
+ --distro-dir=DIR path to distro specific files (debian/RPM)
+ --build-dir=DIR path to dist dir
+ --prefix=PREFIX path to install into
+
+ Optional options:
+ --doc-dir=DIR path to install docs into [/usr/share/doc/elasticsearch]
+ --lib-dir=DIR path to install bits [/usr/lib/elasticsearch]
+ --installed-lib-dir=DIR path where lib-dir will end up on target system
+ --bin-dir=DIR path to install bins [/usr/bin]
+ --examples-dir=DIR path to install examples [doc-dir/examples]
+ ... [ see source for more similar options ]
+ "
+ exit 1
+}
+
+OPTS=$(getopt \
+ -n $0 \
+ -o '' \
+ -l 'prefix:' \
+ -l 'distro-dir:' \
+ -l 'doc-dir:' \
+ -l 'lib-dir:' \
+ -l 'installed-lib-dir:' \
+ -l 'bin-dir:' \
+ -l 'initd-dir:' \
+ -l 'build-dir:' -- "$@")
+
+if [ $? != 0 ] ; then
+ usage
+fi
+
+eval set -- "$OPTS"
+while true ; do
+ case "$1" in
+ --prefix)
+ PREFIX=$2 ; shift 2
+ ;;
+ --distro-dir)
+ DISTRO_DIR=$2 ; shift 2
+ ;;
+ --build-dir)
+ BUILD_DIR=$2 ; shift 2
+ ;;
+ --doc-dir)
+ DOC_DIR=$2 ; shift 2
+ ;;
+ --lib-dir)
+ LIB_DIR=$2 ; shift 2
+ ;;
+ --)
+ shift; break
+ ;;
+ *)
+ echo "Unknown option: $1"
+ usage
+ exit 1
+ ;;
+ esac
+done
+
+for var in PREFIX BUILD_DIR DISTRO_DIR ; do
+ if [ -z "$(eval "echo \$$var")" ]; then
+ echo Missing param: $var
+ usage
+ fi
+done
+
+MAN_DIR=${MAN_DIR:-/usr/share/man/man1}
+DOC_DIR=${DOC_DIR:-/usr/share/doc/kibana}
+LIB_DIR=${LIB_DIR:-/usr/lib/kibana}
+BIN_DIR=${BIN_DIR:-/usr/bin}
+ETC_DIR=${ETC_DIR:-/etc/kibana}
+CONF_DIR=${CONF_DIR:-${ETC_DIR}/conf.dist}
+INITD_DIR=${INITD_DIR:-/etc/init.d}
+
+VAR_DIR=$PREFIX/var
+
+install -d -m 0755 $PREFIX/$MAN_DIR
+install -d -m 0755 $PREFIX/$DOC_DIR
+install -d -m 0755 $PREFIX/$LIB_DIR
+install -d -m 0755 $PREFIX/$ETC_DIR
+install -d -m 0755 $PREFIX/$CONF_DIR
+install -d -m 0755 $PREFIX/$BIN_DIR
+
+# Extract files and copy to LIB_DIR
+tar zxf $BUILD_DIR/kibana-*-linux-*64.tar.gz -C $PREFIX/$LIB_DIR --strip-components 1
Review comment:
I encountered an error here on Ubuntu 18.04:
```
+ tar zxf build//kibana-*-linux-*64.tar.gz -C debian/tmp//usr/lib/kibana --strip-components 1
tar (child): build//kibana-*-linux-*64.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
```
Those files seem to be in a different directory:
```
$ find . -name 'kibana-*-linux-*64.tar.gz'
./output/kibana/kibana-5.4.1/target/kibana-5.4.1-linux-x86_64.tar.gz
./output/kibana/kibana-5.4.1/target/kibana-5.4.1-linux-arm64.tar.gz
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi edited a comment on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
guyuqi edited a comment on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-674806129
Removed unused sysV-init file and disable sysV kibana service.
kibana depends on Elasticsearch. If elasticsearch doesn't start, kibaba would fail to start.
So users would start kibana manually by `$KIBANA_HOME//bin/start-kibana` after they deployed their elasticsearch cluster.
The kibana deploy configuration : `bigtop-deploy/puppet/modules/kibana/`
And this PR leverages it to delpoy a container to run the smoke tests as a separate uses case.
>>On Debian 9 and 10, I failed to install the package with the following error. I tried to add write permission to /usr/lib/kibana/optimize recursively, but the preceding error was not resolved.
I'm sorry I could not reproduce it.
I installed the Deb package on Debian-9 : `dpkg -i kibana_5.4.1-1_all.deb`
The smoke test on debian9:
```
docker:
memory_limit: "4g"
image: "bigtop/puppet:trunk-debian-9"
repo: "http://repos.bigtop.apache.org/releases/1.4.0/debian/9/$(ARCH)"
distro: debian
components: [bigtop-utils, elasticsearch, kibana]
enable_local_repo: true
smoke_test_components: [kibana]
```
`./docker-hadoop.sh -C config_debian-9.yaml -c 1 -s`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi commented on pull request #657: BIGTOP-3320: Add Kibana as Bigtop component
Posted by GitBox <gi...@apache.org>.
guyuqi commented on pull request #657:
URL: https://github.com/apache/bigtop/pull/657#issuecomment-679872117
>It seems that the Source3: kibana.default line in kibana.spec should be removed.
Yes, I'll remove it and rebase this pr for conflicting file: `bigtop_toolchain/manifests/packages.pp`, thanks, @sekikn
>I also tried to run the smoke test on Debian and Ubuntu, but on those distros, start-kibana in TestKibanaSmoke.groovy didn't seem to return and the test didn't finish. Is it the same for you, or it normally finishes on your environment?
I re-verified this PR on Ubuntu16.04 and debian-9 and also succesfully run the smoke tests.
Before running the kibana smoke tests, `elasticsearch `is required to be installed in advance.
Could you please rebuild elasticsearch and kibana again:
1. `docker run --rm -v `pwd`:/ws --workdir /ws bigtop/slaves:trunk-ubuntu-16.04 bash -c '. /etc/profile.d/bigtop.sh; ./gradlew allclean elasticsearch-pkg kibana-pkg apt'`
2. Deploy elasticsearch and kibana in advance:
BIGTOP_HOME/provisioner/docker/config_ubuntu-16.04.yaml:
```
docker:
memory_limit: "4g"
image: "bigtop/puppet:trunk-ubuntu-16.04"
repo: "http://repos.bigtop.apache.org/releases/1.4.0/ubuntu/16.04/$(ARCH)"
distro: debian
components: [elasticsearch, kibana]
enable_local_repo: true
smoke_test_components: [kibana]
```
config_debian-9.yaml
```
docker:
memory_limit: "4g"
image: "bigtop/puppet:trunk-debian-9"
repo: "http://repos.bigtop.apache.org/releases/1.4.0/debian/9/$(ARCH)"
distro: debian
components: [elasticsearch, kibana]
enable_local_repo: true
smoke_test_components: [kibana]
```
3. Run smoke tests:
Unbuntu 16.04:
`./docker-hadoop.sh -C config_ubuntu-16.04 -c 1 -s`
Debian 9:
`./docker-hadoop.sh -C config_debian-9.yaml -c 1 -s
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org