You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by marcotagliabue <gi...@git.apache.org> on 2017/06/22 13:22:22 UTC
[GitHub] zeppelin pull request #2429: [ZEPPELIN-2675] Distributing Jars when using an...
GitHub user marcotagliabue opened a pull request:
https://github.com/apache/zeppelin/pull/2429
[ZEPPELIN-2675] Distributing Jars when using an external flink cluster
### What is this PR for?
This PR intends to make Flink interpreter able to distribute external dependencies on cluster when they are loaded by Web UI.
The code simply collects jar paths downloaded by DependecyResolver and add them to FlinkILoop constructor.
Loading external dependencies in Flink interpreter only work with MiniCluster and Flink version lower than 1.3.0. Only Spark is able to distribute jars at the moment.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2675
### How should this be tested?
1. [Download Flink](https://flink.apache.org/downloads.html) and run local cluster with following command:
```./bin/start-cluster.sh```
2. go to the interpreter page and in dependencies section add artifact:
```joda-time:joda-time:jar:2.9.9```
3. change `local` with `localhost` in interpreter page
4. run below code in paragraph
```
%flink
import org.joda.time.{DateTime, DateTimeZone}
val text = benv.fromElements("To be or not to be")
text
.flatMap { _.toLowerCase.split(" ") }
.map(word => (word, new DateTime(System.currentTimeMillis(), DateTimeZone.UTC) ))
.print()
```
## Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/radicalbit/zeppelin feature/distribute-jars-flink
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2429.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2429
----
commit 711dd80ca05913a3fc0b2e08da569f1c8c4f65df
Author: marcotagliabue <ma...@icloud.com>
Date: 2017-06-22T12:21:28Z
Enable distribution of jars in Flink cluster
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2429: [ZEPPELIN-2675] Distributing Jars when using an extern...
Posted by marcotagliabue <gi...@git.apache.org>.
Github user marcotagliabue commented on the issue:
https://github.com/apache/zeppelin/pull/2429
Hi! I've just updated branch. CI Should be ok. Thanks for the review! @1ambda
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2429: [ZEPPELIN-2675] Distributing Jars when using an extern...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2429
Thanks for the quick reply.
There are 2 failed jobs. We can restart or need to fix them.
https://travis-ci.org/radicalbit/zeppelin/builds/250680464
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2429: [ZEPPELIN-2675] Distributing Jars when using an extern...
Posted by marcotagliabue <gi...@git.apache.org>.
Github user marcotagliabue commented on the issue:
https://github.com/apache/zeppelin/pull/2429
I restarted build in Travis and now is ok. The problem seems to be flaky tests.
@1ambda https://travis-ci.org/radicalbit/zeppelin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2429: [ZEPPELIN-2675] Distributing Jars when using an extern...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2429
LGTM, merge if no more discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2429: [ZEPPELIN-2675] Distributing Jars when using an extern...
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2429
Hi! Thanks for the contribution. @marcotagliabue.
It looks like useful!
1. Please fix the failed CI
2. Would be nice to rebase to update the outdated base branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2429: [ZEPPELIN-2675] Distributing Jars when using an...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/2429
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---