You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by rmetzger <gi...@git.apache.org> on 2016/04/21 21:46:14 UTC
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
GitHub user rmetzger opened a pull request:
https://github.com/apache/bigtop/pull/101
[BIGTOP-2345] Add deb and rpm packaging for Apache Flink
This PR is based on the work done in https://github.com/apache/bigtop/pull/93.
I've picked only the parts regarding the packaging (ignoring the puppet deployment).
The changes have been tested on a Debian8 virtual machine and Fedora 23.
Packaging, installation, the init scripts, user creation, file removal, logging seem to work now.
I'm looking forward to any comments regarding the proposed changes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/bigtop bigtop2345-pr
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bigtop/pull/101.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #101
----
commit f076772ba20ad5a9590c479987e95292e930a752
Author: bhupendrasingh <bh...@gmail.com>
Date: 2016-02-20T21:47:27Z
BIGTOP-2345 Initial prototyping for Flink packaging
commit 1e67d973bf36644e3e13fb5a287b36b72ab2d120
Author: Robert Metzger <rm...@apache.org>
Date: 2016-04-20T15:59:54Z
BIGTOP-2345 Refined Flink packaging
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214239026
I verified the RPM build again. It should be working now.
I'll now test the debian packaging again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214785025
Thank you for the review @mxm!
Regarding the lsb dependency: I saw that the Hadoop spec for example has the following entry:
```
Requires: sh-utils, /lib/lsb/init-functions
```
I guess we can add that as well.
> I wonder, should we rename the service to flink-master and flink-worker? Or flink-jobmanager and flink-taskmanager? Simply TaskManager and JobManager could confuse people.
I'm definitively against `flink-master`, `flink-worker`, because that's not an "official" terminology (at least in the Flink code we call them job- and taskmanager).
I also thought about `flink-jobmanager` and `flink-taskmanager`. Any other thoughts on this?
> Another thing, the logs are named "flink-flink-jobmanager-0-localhost.localdomain.log". Probably could remove the flink-flink prefix.
The naming schema for logs is `flink-<username>-<component>-<instance>-<host>.<log|out>`. By starting the flink services with the `flink` user, we end up with that ugliness.
I'll look into it and try to see if I can change the behavior easily (the problem is that I need to create patches if I want to change anything on the flink source, and I want to avoid patching too much for bigtop).
@oflebbe: I'll check the guide again and adopt the patch integration accordingly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by oflebbe <gi...@git.apache.org>.
Github user oflebbe commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214508154
Hi. My Nitty picky comment:
Please see https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Packaging on how to patch sources. (you may have a look at the hive or zookeeper packages for examples)
Short : rename 1837.patch to patch1-1837.patch . Insert the magic patterns into the spec file. remove the patch command from the do-component-build.
In order to get committed the patch on the branch has to be collapsed to only one patch with the exact message as the JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2345 Create Flink packaging
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger closed the pull request at:
https://github.com/apache/bigtop/pull/101
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2345 Create Flink packaging
Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-217156713
Hey guys, any update on this? I am looking forward to having the Flink integration in. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/101#discussion_r60841510
--- Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec ---
@@ -0,0 +1,143 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+%define flink_name flink
+%define lib_flink /usr/lib/%{flink_name}
+%define bin_flink /usr/bin
+%define etc_flink /etc/%{flink_name}
+%define config_flink %{etc_flink}/conf
+%define man_dir %{_mandir}
+%define flink_services jobmanager taskmanager
+%define var_run_flink /var/run/%{flink_name}
+%define var_log_flink /var/log/%{flink_name}
+
+
+%if %{!?suse_version:1}0
+%define doc_flink %{_docdir}/%{flink_name}-%{flink_version}
+%define alternatives_cmd alternatives
+%define build_flink %{_builddir}/%{flink_name}-%{flink_version}/flink-dist/target/%{flink_name}-%{flink_version}-bin/%{flink_name}-%{flink_version}/
+
+%else
+%define doc_flink %{_docdir}/%{flink_name}-%{flink_version}
+%define alternatives_cmd update-alternatives
+%endif
+
+Name: %{flink_name}
+Version: %{flink_version}
+Release: %{flink_release}
+Summary: Apache Flink is an open source platform for distributed stream and batch data processing.
+License: ASL 2.0
+URL: http://flink.apache.org/
+Group: Development/Libraries
+Buildroot: %{_topdir}/INSTALL/%{name}-%{version}
+BuildArch: noarch
+Source0: flink-%{flink_base_version}.tar.gz
+Source1: do-component-build
+Source2: install_flink.sh
+Source3: init.d.tmpl
+Source4: jobmanager.svc
+Source5: taskmanager.svc
+Source6: bigtop.bom
+Requires: bigtop-utils >= 0.7
+Requires(preun): /sbin/service
+
+%description
+Apache Flink is an open source platform for distributed stream and batch data processing.
+Flink’s core is a streaming dataflow engine that provides data distribution, communication,
+and fault tolerance for distributed computations over data streams.
+
+Flink includes several APIs for creating applications that use the Flink engine:
+ * DataStream API for unbounded streams embedded in Java and Scala, and
+ * DataSet API for static data embedded in Java, Scala, and Python,
+ * Table API with a SQL-like expression language embedded in Java and Scala.
+
+Flink also bundles libraries for domain-specific use cases:
+ * Machine Learning library, and
+ * Gelly, a graph processing API and library.
+
+Some of the key features of Apache Flink includes.
+ * Complete Event Processing (CEP)
+ * Fault-tolerance via Lightweight Distributed Snapshots
+ * Hadoop-native YARN & HDFS implementation
+
+# Additions for master-worker configuration #
+
+%global initd_dir %{_sysconfdir}/init.d
+
+%if %{?suse_version:1}0
+# Required for init scripts
+Requires: insserv
+%global initd_dir %{_sysconfdir}/rc.d
+
+%else
+# Required for init scripts
+Requires: /lib/lsb/init-functions
+%global initd_dir %{_sysconfdir}/rc.d/init.d
+%endif
+
+##############################################
+
+%prep
+%setup -n %{name}-%{flink_base_version}
+
+%build
+bash $RPM_SOURCE_DIR/do-component-build
+
+
+
+# Init.d scripts
+%__install -d -m 0755 $RPM_BUILD_ROOT/%{initd_dir}/
+
+%install
+%__rm -rf $RPM_BUILD_ROOT
+
+sh -x %{SOURCE2} --prefix=$RPM_BUILD_ROOT --source-dir=$RPM_SOURCE_DIR --build-dir=`pwd`/build-target
+
+
+
+for service in %{flink_services}
+do
+ # Install init script
+ init_file=$RPM_BUILD_ROOT/%{initd_dir}/%{flink_name}-${service}
+ bash %{SOURCE3} $RPM_SOURCE_DIR/%{flink_name}-${service}.svc rpm $init_file
+done
+
+
+%preun
+for service in %{flink_services}; do
+ /sbin/service %{flink_name}-${service} status > /dev/null 2>&1
+ if [ $? -eq 0 ]; then
+ /sbin/service %{flink_name}-${service} stop > /dev/null 2>&1
+ fi
+done
+
+
+%post
+%{alternatives_cmd} --install %{config_flink} %{flink_name}-conf %{config_flink}.dist 30
+
+###### FILES ###########
+
+%files
+%defattr(-,root,root,755)
+%config(noreplace) %{config_flink}.dist
+
+%dir %{_sysconfdir}/%{flink_name}
+%config(noreplace) %{initd_dir}/%{flink_name}-master
+%config(noreplace) %{initd_dir}/%{flink_name}-worker
--- End diff --
It seems that the rebasing before the PR went horribly wrong :(
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2345 Create Flink packaging
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-217852730
Thanks a lot for merging.
I'm closing the PR now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214802211
>I'm definitively against flink-master, flink-worker, because that's not an "official" terminology (at least in the Flink code we call them job- and taskmanager).
Then let's call them `flink-jobmanager` and `flink-taskmanager`. The `flink-` prefix will help people to discover the service names.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-215009947
Okay, I'll rename the services
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2345 Create Flink packaging
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-215436502
I addressed all remaining issues with the pull request.
Please let me know if there's anything else blocking this from being merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-213452902
I'm getting the following error message when building using `gradle flink-rpm`:
>RPM build errors:
File not found: /home/max/Desktop/bigtop/build/flink/rpm/BUILDROOT/flink-1.0.0-1.fc23.x86_64/etc/rc.d/init.d/flink-master
File not found: /home/max/Desktop/bigtop/build/flink/rpm/BUILDROOT/flink-1.0.0-1.fc23.x86_64/etc/rc.d/init.d/flink-worker
File not found: /home/max/Desktop/bigtop/build/flink/rpm/BUILDROOT/flink-1.0.0-1.fc23.x86_64/usr/share/doc/flink-1.0.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214249097
Debian is now fixed as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/101#discussion_r60749429
--- Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec ---
@@ -0,0 +1,143 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+%define flink_name flink
+%define lib_flink /usr/lib/%{flink_name}
+%define bin_flink /usr/bin
+%define etc_flink /etc/%{flink_name}
+%define config_flink %{etc_flink}/conf
+%define man_dir %{_mandir}
+%define flink_services jobmanager taskmanager
+%define var_run_flink /var/run/%{flink_name}
+%define var_log_flink /var/log/%{flink_name}
+
+
+%if %{!?suse_version:1}0
+%define doc_flink %{_docdir}/%{flink_name}-%{flink_version}
+%define alternatives_cmd alternatives
+%define build_flink %{_builddir}/%{flink_name}-%{flink_version}/flink-dist/target/%{flink_name}-%{flink_version}-bin/%{flink_name}-%{flink_version}/
+
+%else
+%define doc_flink %{_docdir}/%{flink_name}-%{flink_version}
+%define alternatives_cmd update-alternatives
+%endif
+
+Name: %{flink_name}
+Version: %{flink_version}
+Release: %{flink_release}
+Summary: Apache Flink is an open source platform for distributed stream and batch data processing.
+License: ASL 2.0
+URL: http://flink.apache.org/
+Group: Development/Libraries
+Buildroot: %{_topdir}/INSTALL/%{name}-%{version}
+BuildArch: noarch
+Source0: flink-%{flink_base_version}.tar.gz
+Source1: do-component-build
+Source2: install_flink.sh
+Source3: init.d.tmpl
+Source4: jobmanager.svc
+Source5: taskmanager.svc
+Source6: bigtop.bom
+Requires: bigtop-utils >= 0.7
+Requires(preun): /sbin/service
+
+%description
+Apache Flink is an open source platform for distributed stream and batch data processing.
+Flink’s core is a streaming dataflow engine that provides data distribution, communication,
+and fault tolerance for distributed computations over data streams.
+
+Flink includes several APIs for creating applications that use the Flink engine:
+ * DataStream API for unbounded streams embedded in Java and Scala, and
+ * DataSet API for static data embedded in Java, Scala, and Python,
+ * Table API with a SQL-like expression language embedded in Java and Scala.
+
+Flink also bundles libraries for domain-specific use cases:
+ * Machine Learning library, and
+ * Gelly, a graph processing API and library.
+
+Some of the key features of Apache Flink includes.
+ * Complete Event Processing (CEP)
+ * Fault-tolerance via Lightweight Distributed Snapshots
+ * Hadoop-native YARN & HDFS implementation
+
+# Additions for master-worker configuration #
+
+%global initd_dir %{_sysconfdir}/init.d
+
+%if %{?suse_version:1}0
+# Required for init scripts
+Requires: insserv
+%global initd_dir %{_sysconfdir}/rc.d
+
+%else
+# Required for init scripts
+Requires: /lib/lsb/init-functions
+%global initd_dir %{_sysconfdir}/rc.d/init.d
+%endif
+
+##############################################
+
+%prep
+%setup -n %{name}-%{flink_base_version}
+
+%build
+bash $RPM_SOURCE_DIR/do-component-build
+
+
+
+# Init.d scripts
+%__install -d -m 0755 $RPM_BUILD_ROOT/%{initd_dir}/
+
+%install
+%__rm -rf $RPM_BUILD_ROOT
+
+sh -x %{SOURCE2} --prefix=$RPM_BUILD_ROOT --source-dir=$RPM_SOURCE_DIR --build-dir=`pwd`/build-target
+
+
+
+for service in %{flink_services}
+do
+ # Install init script
+ init_file=$RPM_BUILD_ROOT/%{initd_dir}/%{flink_name}-${service}
+ bash %{SOURCE3} $RPM_SOURCE_DIR/%{flink_name}-${service}.svc rpm $init_file
+done
+
+
+%preun
+for service in %{flink_services}; do
+ /sbin/service %{flink_name}-${service} status > /dev/null 2>&1
+ if [ $? -eq 0 ]; then
+ /sbin/service %{flink_name}-${service} stop > /dev/null 2>&1
+ fi
+done
+
+
+%post
+%{alternatives_cmd} --install %{config_flink} %{flink_name}-conf %{config_flink}.dist 30
+
+###### FILES ###########
+
+%files
+%defattr(-,root,root,755)
+%config(noreplace) %{config_flink}.dist
+
+%dir %{_sysconfdir}/%{flink_name}
+%config(noreplace) %{initd_dir}/%{flink_name}-master
+%config(noreplace) %{initd_dir}/%{flink_name}-worker
--- End diff --
I think those need to be
````
%config(noreplace) %{initd_dir}/%{flink_name}-jobmanager
%config(noreplace) %{initd_dir}/%{flink_name}-taskmanager
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214375936
Tested the RPM package on Fedora 23. First, a bit of a disappointment:
```bash
● jobmanager.service - LSB: Flink jobmanager
Loaded: loaded (/etc/rc.d/init.d/jobmanager)
Active: failed (Result: exit-code) since Mon 2016-04-25 16:40:45 CEST; 16s ago
Docs: man:systemd-sysv-generator(8)
Process: 13344 ExecStart=/etc/rc.d/init.d/jobmanager start (code=exited, status=127)
Apr 25 16:40:41 localhost.localdomain jobmanager[13344]: /etc/rc.d/init.d/jobmanager: line 34: /lib/lsb/init-functions: No such file or directory
Apr 25 16:40:41 localhost.localdomain jobmanager[13344]: /etc/rc.d/init.d/jobmanager: line 74: log_success_msg: command not found
Apr 25 16:40:41 localhost.localdomain jobmanager[13344]: /etc/rc.d/init.d/jobmanager: line 120: pidofproc: command not found
Apr 25 16:40:41 localhost.localdomain su[13386]: (to flink) root on none
Apr 25 16:40:45 localhost.localdomain jobmanager[13344]: /etc/rc.d/init.d/jobmanager: line 120: pidofproc: command not found
Apr 25 16:40:45 localhost.localdomain jobmanager[13344]: /etc/rc.d/init.d/jobmanager: line 102: log_failure_msg: command not found
Apr 25 16:40:45 localhost.localdomain systemd[1]: jobmanager.service: Control process exited, code=exited status=127
Apr 25 16:40:45 localhost.localdomain systemd[1]: Failed to start LSB: Flink jobmanager.
Apr 25 16:40:45 localhost.localdomain systemd[1]: jobmanager.service: Unit entered failed state.
Apr 25 16:40:45 localhost.localdomain systemd[1]: jobmanager.service: Failed with result 'exit-code'.
```
Turns out, it works when you install `lsb` or more specifically `redhat-lsb-core`:
```bash
● jobmanager.service - LSB: Flink jobmanager
Loaded: loaded (/etc/rc.d/init.d/jobmanager)
Active: active (exited) since Mon 2016-04-25 16:42:56 CEST; 2s ago
Docs: man:systemd-sysv-generator(8)
Process: 14071 ExecStart=/etc/rc.d/init.d/jobmanager start (code=exited, status=0/SUCCESS)
Apr 25 16:42:56 localhost.localdomain systemd[1]: Starting LSB: Flink jobmanager...
Apr 25 16:42:56 localhost.localdomain jobmanager[14071]: Starting Flink jobmanager (flink-jobmanager):[ OK ]
Apr 25 16:42:56 localhost.localdomain jobmanager[14071]: Flink jobmanager is running[ OK ]
Apr 25 16:42:56 localhost.localdomain systemd[1]: Started LSB: Flink jobmanager.
● taskmanager.service - LSB: Flink taskmanager
Loaded: loaded (/etc/rc.d/init.d/taskmanager)
Active: active (exited) since Mon 2016-04-25 16:44:28 CEST; 22s ago
Docs: man:systemd-sysv-generator(8)
Process: 15428 ExecStart=/etc/rc.d/init.d/taskmanager start (code=exited, status=0/SUCCESS)
Apr 25 16:44:25 localhost.localdomain systemd[1]: Starting LSB: Flink taskmanager...
Apr 25 16:44:25 localhost.localdomain taskmanager[15428]: Starting Flink taskmanager (flink-taskmanager):[ OK ]
Apr 25 16:44:25 localhost.localdomain su[15483]: (to flink) root on none
Apr 25 16:44:28 localhost.localdomain taskmanager[15428]: Started Flink taskmanager (flink-taskmanager):[ OK ]
Apr 25 16:44:28 localhost.localdomain systemd[1]: Started LSB: Flink taskmanager.
```
So we need to add `lsb` as a dependency for Flink. Or do we assume people have it installed?
The jobmanager/taskmanager came up fine. I wonder, should we rename the service to `flink-master` and `flink-worker`? Or `flink-jobmanager` and `flink-taskmanager`? Simply TaskManager and JobManager could confuse people.
Another thing, the logs are named "flink-flink-jobmanager-0-localhost.localdomain.log". Probably could remove the `flink-flink` prefix.
Other than that, great work :+1:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214794072
@oflebbe quick question:
The spec file contains already the following section:
```spec
%if %{?suse_version:1}0
# Required for init scripts
Requires: insserv
%global initd_dir %{_sysconfdir}/rc.d
%else
# Required for init scripts
Requires: /lib/lsb/init-functions
%global initd_dir %{_sysconfdir}/rc.d/init.d
%endif
```
I assume the if condition is true for suse, so the else branch applied for Max' build, requiring the init-functions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214720306
Thank you for the feedback!
I'm trying to address them soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214798709
Another follow up Q:
> You should not package /var/run, since this is only a temporary filesystem. You should create it with your init script (This will fix one of the lintian errors for debian/ubuntu).
Okay, makes sense. I was looking a bit into other scripts and I saw that others are setting the `$WORKING_DIR` variable.
I wonder why `/bigtop-packages/src/templates/init.d.tmpl` is not taking care of the creation and deletion of the `$WORKING_DIR` for that?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by bhupendrasingh <gi...@git.apache.org>.
Github user bhupendrasingh commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/101#discussion_r60838552
--- Diff: bigtop-packages/src/rpm/flink/SPECS/flink.spec ---
@@ -0,0 +1,145 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+%define flink_name flink
+%define lib_flink /usr/lib/%{flink_name}
+%define bin_flink /usr/bin
+%define etc_flink /etc/%{flink_name}
+%define config_flink %{etc_flink}/conf
+%define man_dir %{_mandir}
+%define flink_services master worker
--- End diff --
I think this also needs to be changed to -
%define flink_services jobmanager taskmanager
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2345 Create Flink packaging
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-217743938
I have committed this to the master. Feel free to close the PR and thanks for your contribution!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-213926942
The gradle build of the RPM should be fixed. But there seems to be still an issue with the systemd start of the jobmanager.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: [BIGTOP-2345] Add deb and rpm packaging for A...
Posted by oflebbe <gi...@git.apache.org>.
Github user oflebbe commented on the pull request:
https://github.com/apache/bigtop/pull/101#issuecomment-214708194
Hi the build looks very promising! You should not package /var/run, since this is only a temporary filesystem. You should create it with your init script (This will fix one of the lintian errors for debian/ubuntu).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---