You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by youngwookim <gi...@git.apache.org> on 2015/08/24 11:45:55 UTC
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
GitHub user youngwookim opened a pull request:
https://github.com/apache/bigtop/pull/32
BIGTOP-1561. Add Presto as a component
https://issues.apache.org/jira/browse/BIGTOP-1561
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/youngwookim/bigtop BIGTOP-1561
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bigtop/pull/32.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #32
----
commit 148c2f50a7237abf7b81abe6eefba6c47c69f098
Author: Youngwoo Kim <yw...@apache.org>
Date: 2015-02-16T11:39:41Z
BIGTOP-1561. Add Presto as a component
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by spew <gi...@git.apache.org>.
Github user spew commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-193497374
On line 211 of install_presto.sh we reference the location of presto-env.sh without including the $CONF_FOLDER directory
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by youngwookim <gi...@git.apache.org>.
Github user youngwookim commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-187688594
@cawallin I'll look into presto-server-rpm and presto-admin. I believe presto-admin is nice so it would be nice to have the compatibility between Bigtop's artifacts and presto-admin.
First of all, packaging rpm on Bigtop is a bit different from presto-server-rpm. overall, we have a base package and sub packages. the spec file for Hadoop is a good example: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec
I applied the rules to Presto's one. so there are base package 'presto' and the rest of the packages are sub package: 'presto-server' for server service, 'presto-cli' for Presto client, 'presto-jdbc' for JDBC driver and 'presto-doc' for generated documentation. usually, we have configurations and init script in the server package.
Looks like presto-server-rpm contains the server package only. I wonder if you have a plan for cli, doc and etc.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by cawallin <gi...@git.apache.org>.
Github user cawallin commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-188324478
Thanks for the explanation! We aren't actively working on the RPM right now, but we have it in our backlog to look into adding the CLI/docs/verifier/benchmark driver to the RPM, though we'd probably have to refactor it to move to the packages/subpackages approach you have here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by cawallin <gi...@git.apache.org>.
Github user cawallin commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/32#discussion_r53353145
--- Diff: bigtop-packages/src/common/presto/presto-server.svc ---
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+TYPE="server"
+DAEMON="presto-${TYPE}"
+DESC="Presto ${TYPE}"
+EXEC_PATH="/usr/lib/presto/bin/launcher"
+SVC_USER="presto"
+WORKING_DIR="/usr/lib/presto"
+DAEMON_FLAGS="--launcher-config /usr/lib/presto/bin/launcher.properties --jvm-config /etc/presto/jvm.config --config /etc/presto/config.properties --data-dir /var/lib/presto --pid-file /var/run/presto/${DAEMON}.pid --launcher-log-file /var/log/presto/${DAEMON}.out --server-log-file /var/log/presto/${DAEMON}.log"
--- End diff --
Not sure if this is relevant for BigTop or not, but it's possible to set pretty much all of the log/data file locations in the Presto node.properties file (e.g. node.launcher-log-file), which wouldn't be picked up by this hard coding of locations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by cawallin <gi...@git.apache.org>.
Github user cawallin commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-185389704
Oh, no; is there anywhere that the Bigtop RPM expectations are documented?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-183613836
Does it also have deb?
Does the layout and the build process for this is done according to the Bigtop expectation (also used by Hadoop distributions at large)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by youngwookim <gi...@git.apache.org>.
Github user youngwookim commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/32#discussion_r53773010
--- Diff: bigtop-packages/src/common/presto/presto-server.svc ---
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+TYPE="server"
+DAEMON="presto-${TYPE}"
+DESC="Presto ${TYPE}"
+EXEC_PATH="/usr/lib/presto/bin/launcher"
+SVC_USER="presto"
+WORKING_DIR="/usr/lib/presto"
+DAEMON_FLAGS="--launcher-config /usr/lib/presto/bin/launcher.properties --jvm-config /etc/presto/jvm.config --config /etc/presto/config.properties --data-dir /var/lib/presto --pid-file /var/run/presto/${DAEMON}.pid --launcher-log-file /var/log/presto/${DAEMON}.out --server-log-file /var/log/presto/${DAEMON}.log"
--- End diff --
@cawallin Good point! Will take a look and update that. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by c0s <gi...@git.apache.org>.
Github user c0s commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-185569957
There's no explicit document on RPM's requirements, but all of our packaging code is following the same rules and informal guidelines, codified within the existing packaging code. Good examples for complex packages would be hadoop and hbase. Lesser complex stuff is ignite-hadoop and similar.
We won't be able to accept _only_ RPM packaging though: half of our supported OS are deb-based.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by youngwookim <gi...@git.apache.org>.
Github user youngwookim commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-185595881
@cawallin I did basic packaging for presto to use our internal deployment and at that time, there is no packages like rpm or deb on Presto. I'm aware of that you and your team (Teradata, right?) are doing great work in Presto community :-) As of now, we have some prerequisites to integrate Presto into Bigtop stack. and... now, I believe it's a good starting point to discuss on that. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by cawallin <gi...@git.apache.org>.
Github user cawallin commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-185824052
Sure, thanks for the update, sorry I was a bit insistent before :) I'm not sure that we'll have room in our roadmap to work on deb packaging, but tools like presto-admin (https://github.com/prestodb/presto-admin) depend a little on how the RPM/init scripts work, and it'd be great if presto-admin worked for both RPMs. After looking more carefully through the code, I think it should work, though I made one comment about configurability (e.g. if you discover after installation that your /var mount is really small and want to change the log directory after the fact), and I'd have to test out the RPM with our presto-admin tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] bigtop pull request: BIGTOP-1561. Add Presto as a component
Posted by cawallin <gi...@git.apache.org>.
Github user cawallin commented on the pull request:
https://github.com/apache/bigtop/pull/32#issuecomment-183514054
Presto has an RPM, it would be best to use that: https://github.com/facebook/presto/tree/master/presto-server-rpm.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---