You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by rmetzger <gi...@git.apache.org> on 2016/05/26 10:26:03 UTC
[GitHub] bigtop pull request: BIGTOP-2357: Add Apache Flink to Bigtop / Cre...
GitHub user rmetzger opened a pull request:
https://github.com/apache/bigtop/pull/111
BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet recipes
With this change, I'm adding some basic puppet scripts for deploying flink.
I've tested the change using the `bigtop-deploy/vm/vagrant-puppet-vm/`. Its the first time that I'm working with puppet, so any feedback is very welcome.
While testing, I also came across https://issues.apache.org/jira/browse/BIGTOP-2157. Please let me know if the fix I'm suggesting in this PR is appropriate.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/bigtop bigtop2357
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bigtop/pull/111.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #111
----
commit ccdb13eaf420214abf0050dd2e38e91c0d037a1c
Author: Robert Metzger <rm...@apache.org>
Date: 2016-05-24T12:51:48Z
BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet recipes
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create p...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger closed the pull request at:
https://github.com/apache/bigtop/pull/111
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet r...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the issue:
https://github.com/apache/bigtop/pull/111
Thank you for the review. I addressed your comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet r...
Posted by rvs <gi...@git.apache.org>.
Github user rvs commented on the issue:
https://github.com/apache/bigtop/pull/111
Great! If nobody has any other concerns -- I'll be merging it in tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create p...
Posted by rvs <gi...@git.apache.org>.
Github user rvs commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/111#discussion_r67065894
--- Diff: bigtop-deploy/puppet/modules/flink/manifests/init.pp ---
@@ -0,0 +1,64 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+class flink {
+
+ class deploy ($roles) {
+ if ("flink-jobmanager" in $roles) {
+ include flink::jobmanager
+ }
+
+ if ("flink-taskmanager" in $roles) {
+ include flink::taskmanager
+ }
+ }
+
+ class common($jobmanager_host, $jobmanager_port, $ui_port) {
+ # make sure flink is installed
+ package { "flink":
+ ensure => latest
+ }
+
+
+ # set values in flink-conf.yaml
+ $tmp_dirs = hadoop::hadoop_storage_dirs
--- End diff --
I'd be much more comfortable if this was passed in as a parameter from the outside of the puppet Flink class. Can you please do it that way?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2357: Add Apache Flink to Bigtop / Cre...
Posted by ktsakalozos <gi...@git.apache.org>.
Github user ktsakalozos commented on the pull request:
https://github.com/apache/bigtop/pull/111#issuecomment-222458643
Thank you for doing this your work. It will be very helpful to us (juju team) since we plan to create a charm for Flink soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet r...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the issue:
https://github.com/apache/bigtop/pull/111
@rvs any chances you can have a look or find somebody who can?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet r...
Posted by rvs <gi...@git.apache.org>.
Github user rvs commented on the issue:
https://github.com/apache/bigtop/pull/111
Will definitely take a look over the weekend once I'm back in US.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet r...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the issue:
https://github.com/apache/bigtop/pull/111
Ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create p...
Posted by rvs <gi...@git.apache.org>.
Github user rvs commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/111#discussion_r67066106
--- Diff: bigtop-deploy/vm/vagrant-puppet-vm/Vagrantfile ---
@@ -112,7 +113,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
bigtop.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", CONF['memory_size']]
- vb.customize ['modifyvm', :id, '--cpus', CONF['number_cpus']]
+ vb.customize ['modifyvm', :id, '--cpus', CONF['number_cpus']]
--- End diff --
This appears to be just a whitespace change - if you'll be fixing the comment above, can you please remove this spurious change as well?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create p...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/111#discussion_r67134651
--- Diff: bigtop-deploy/vm/vagrant-puppet-vm/Vagrantfile ---
@@ -112,7 +113,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
bigtop.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", CONF['memory_size']]
- vb.customize ['modifyvm', :id, '--cpus', CONF['number_cpus']]
+ vb.customize ['modifyvm', :id, '--cpus', CONF['number_cpus']]
--- End diff --
Sure, I will undo the change. Its not an accidental change. The file is using spaces for indentation, and only this line uses tabs (which makes it look unaligned in some editors)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-2357: Add Apache Flink to Bigtop / Cre...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/bigtop/pull/111#issuecomment-222460271
Great, looking forward to having a charm for Flink.
Please let me know if you find any issues with these puppet recipes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop issue #111: BIGTOP-2357: Add Apache Flink to Bigtop / Create puppet r...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the issue:
https://github.com/apache/bigtop/pull/111
Great, thank you ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---