You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by HeartSaVioR <gi...@git.apache.org> on 2016/07/21 06:46:27 UTC
[GitHub] storm pull request #1581: STORM-1993 Update storm-sql README to have actual ...
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1581
STORM-1993 Update storm-sql README to have actual dependencies
Please refer [STORM-1993](https://issues.apache.org/jira/browse/STORM-1993) for more details.
Tested with 1.1.0 RC1 and 1.0.2 RC3. Should also work on master since 2.x and 1.x should have same dependencies for now.
Can also be ported back to 1.x-branch and 1.0.x-branch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-1993
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1581.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1581
----
commit 4b91a1ee245f2971ba6712ac77a1278931d2f4ad
Author: Jungtaek Lim <ka...@gmail.com>
Date: 2016-07-21T06:41:51Z
STORM-1993 Update storm-sql README to have actual dependencies
* there're many missing dependencies
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1581: STORM-1993 Update storm-sql README to have actual ...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR closed the pull request at:
https://github.com/apache/storm/pull/1581
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1581: STORM-1993 Update storm-sql README to have actual depende...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1581
This document is only valid for 1.0.x branch. In master and 1.x-branch we don't need to do this to run sql.
This is a documentation change so I'll just merge this without +1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1581: STORM-1993 Update storm-sql README to have actual ...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1581#discussion_r75819500
--- Diff: external/sql/README.md ---
@@ -70,17 +70,31 @@ To run this example, users need to include the data sources (`storm-sql-kafka` i
class path. One approach is to put the required jars into the `extlib` directory:
```
-$ cp curator-client-2.5.0.jar curator-framework-2.5.0.jar zookeeper-3.4.6.jar
- extlib/
-$ cp scala-library-2.10.4.jar kafka-clients-0.8.2.1.jar kafka_2.10-0.8.2.1.jar metrics-core-2.2.0.jar extlib/
-$ cp json-simple-1.1.1.jar extlib/
-$ cp storm-kafka-*.jar storm-sql-kafka-*.jar storm-sql-runtime-*.jar extlib/
+calcite-avatica-1.4.0-incubating.jar
--- End diff --
@manuzhang
Since both STORM-2016 and STORM-2023 are merged into master and 1.x branch, we can run this for fairly simple command. I'd rather close this and update the doc just for 1.1.0 and 2.0.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1581: STORM-1993 Update storm-sql README to have actual ...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1581#discussion_r72003648
--- Diff: external/sql/README.md ---
@@ -70,17 +70,31 @@ To run this example, users need to include the data sources (`storm-sql-kafka` i
class path. One approach is to put the required jars into the `extlib` directory:
```
-$ cp curator-client-2.5.0.jar curator-framework-2.5.0.jar zookeeper-3.4.6.jar
- extlib/
-$ cp scala-library-2.10.4.jar kafka-clients-0.8.2.1.jar kafka_2.10-0.8.2.1.jar metrics-core-2.2.0.jar extlib/
-$ cp json-simple-1.1.1.jar extlib/
-$ cp storm-kafka-*.jar storm-sql-kafka-*.jar storm-sql-runtime-*.jar extlib/
+calcite-avatica-1.4.0-incubating.jar
--- End diff --
Unfortunately you need to find them from maven central and add them to extlib. Note that they'll be added to worker classpath for all workers on cluster.
I know it's really bad, so I'm thinking about how to resolve this.
For now I'm working on adding dependencies from submission step.
That would be similar (nearly same) to --jars and --packages for spark-submit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1581: STORM-1993 Update storm-sql README to have actual ...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/storm/pull/1581#discussion_r72002584
--- Diff: external/sql/README.md ---
@@ -70,17 +70,31 @@ To run this example, users need to include the data sources (`storm-sql-kafka` i
class path. One approach is to put the required jars into the `extlib` directory:
```
-$ cp curator-client-2.5.0.jar curator-framework-2.5.0.jar zookeeper-3.4.6.jar
- extlib/
-$ cp scala-library-2.10.4.jar kafka-clients-0.8.2.1.jar kafka_2.10-0.8.2.1.jar metrics-core-2.2.0.jar extlib/
-$ cp json-simple-1.1.1.jar extlib/
-$ cp storm-kafka-*.jar storm-sql-kafka-*.jar storm-sql-runtime-*.jar extlib/
+calcite-avatica-1.4.0-incubating.jar
--- End diff --
where should I look for these jar files ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1581: STORM-1993 Update storm-sql README to have actual depende...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1581
Cherry picked via 8bc442563326a29bbeab9b2d1c486d4ec11f9f00
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1581: STORM-1993 Update storm-sql README to have actual ...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1581#discussion_r73668059
--- Diff: external/sql/README.md ---
@@ -70,17 +70,31 @@ To run this example, users need to include the data sources (`storm-sql-kafka` i
class path. One approach is to put the required jars into the `extlib` directory:
```
-$ cp curator-client-2.5.0.jar curator-framework-2.5.0.jar zookeeper-3.4.6.jar
- extlib/
-$ cp scala-library-2.10.4.jar kafka-clients-0.8.2.1.jar kafka_2.10-0.8.2.1.jar metrics-core-2.2.0.jar extlib/
-$ cp json-simple-1.1.1.jar extlib/
-$ cp storm-kafka-*.jar storm-sql-kafka-*.jar storm-sql-runtime-*.jar extlib/
+calcite-avatica-1.4.0-incubating.jar
--- End diff --
@manuzhang
Please refer https://github.com/apache/storm/pull/1608#issuecomment-237807425
Once we merge STORM-2016 we can get rid of needs for copying dependencies to extlib. When STORM-2023 is merged, running `storm sql` will be much simpler.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1581: STORM-1993 Update storm-sql README to have actual depende...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1581
This is just a workaround before addressing STORM-1435.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---