You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Leemoonsoo <gi...@git.apache.org> on 2015/05/18 05:32:56 UTC
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
GitHub user Leemoonsoo opened a pull request:
https://github.com/apache/incubator-zeppelin/pull/75
ZEPPELIN-44 Interpreter for Apache Flink
#### Interpreter for [Apache Flink](http://flink.apache.org/).
Flink people helped a lot to write the interpreter. Thanks so much! Some codes are copied from Flink's development branch. Once Flink releases 0.9, copied code and snapshot repository configuration will be removed.
#### Build
if there're no options, by default it is building against flink 0.9.0-milestone-1.
With combination of Zeppelin, it is good idea to use 0.9-SNAPSHOT, because of it support .collect() that helps really a lot to get results data and display it on Zeppelin.
So, you might want to build in this way,
```
mvn package -Dflink.version=0.9-SNAPSHOT -DskipTests
```
#### Screenshot
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Leemoonsoo/incubator-zeppelin flink
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/75.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #75
----
commit c0d2b9d5c34e7560e46c7fcb1a43d0ee1eb8d924
Author: Lee moon soo <mo...@apache.org>
Date: 2015-05-11T14:56:00Z
Initial implementation of interpreter for Apache Flink
commit 9a7a6a38d6e6ec83e00490fa01a146b692316ab3
Author: Lee moon soo <mo...@apache.org>
Date: 2015-05-11T17:43:52Z
Cleaning up
commit f255a036f671aa6643fe02124856e2958214645d
Author: Lee moon soo <mo...@apache.org>
Date: 2015-05-11T17:55:53Z
Fix unittest and update comment
commit 0ca3a4762661d5d70a556fcb47e297ea8eb031bf
Author: Lee moon soo <mo...@apache.org>
Date: 2015-05-18T03:23:24Z
Add apache snapshot repo
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/75
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104193702
I forwarded the error to the Flink dev mailing list. Hopefully somebody from the Flink community can help with this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-105445614
@rmetzger Not blocked by anything from Flink side.
I'm testing the branch with 9 node Flink cluster environment. Works well but looks like need special build command. I'll update build configuration.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-110160168
Merging it if there is no more discussions :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by hsaputra <gi...@git.apache.org>.
Github user hsaputra commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-110837103
W000t!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-109906811
After resolving Zeppelin side issue by #88, Tests are passing.
Ready to merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-105443702
Okay, so this pull request is not blocked by anything from the Flink side right now?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-102916798
@Leemoonsoo Wow, Does it work?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter
for Apach...
Posted by Stephan Ewen <se...@apache.org>.
True, may still have another issue.
On Thu, May 21, 2015 at 1:45 PM, Till Rohrmann <tr...@apache.org> wrote:
> Hmm that is interesting. I always thought that in the case of multiple
> SLF4J bindings, slf4j will simply pick one of them (first, last, random).
>
> On Thu, May 21, 2015 at 12:34 PM, Stephan Ewen <se...@apache.org> wrote:
>
> > The actor system fails to start. Here are the suspicious log lines:
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in [jar:file:/home/travis/.m2/
> > repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-
> > 1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/home/travis/build/
> > apache/incubator-zeppelin/zeppelin-interpreter/target/
> > lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> > explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> >
> > Seems that SLF4J has more than one binding. I remember that Akka crashed
> > because of that before.
> >
> > As a simple fix, can you try and exclude the SLF4J jar from your build
> > somehow? Or set it to "provided" in the Flink POM?
> >
> >
> >
> >
> >
> > On Thu, May 21, 2015 at 11:49 AM, Till Rohrmann <tr...@apache.org>
> > wrote:
> >
> > > I'll try to reproduce this problem locally on my machine.
> > >
> > > On Thu, May 21, 2015 at 11:25 AM, Fabian Hueske <fh...@gmail.com>
> > wrote:
> > >
> > > > Hi Flink folks,
> > > >
> > > > the Flink interpreter PR for Apache Zeppelin is blocked by a failing
> > test
> > > > case (see below).
> > > > Does anybody have an idea what is going on and can maybe help to
> > resolve
> > > > the problem?
> > > >
> > > > Thanks, Fabian
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Leemoonsoo <gi...@git.apache.org>
> > > > Date: 2015-05-21 6:35 GMT+02:00
> > > > Subject: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44
> > > Interpreter
> > > > for Apach...
> > > > To: dev@zeppelin.incubator.apache.org
> > > >
> > > >
> > > > Github user Leemoonsoo commented on the pull request:
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104132384
> > > >
> > > > This PR has some test using FlinkMiniCluster.
> > > > Tests are fine when i run them on my IDE.
> > > > But if i run them using maven, it fails. (CI is failing, too)
> > > >
> > > > I think everything is ready, except for this test failure. Any
> > advice
> > > > is very appreciated.
> > > > here's failure message
> > > >
> > > > ```
> > > > -------------------------------------------------------
> > > > T E S T S
> > > > -------------------------------------------------------
> > > > Running org.apache.zeppelin.flink.FlinkInterpreterTest
> > > > SLF4J: Class path contains multiple SLF4J bindings.
> > > > SLF4J: Found binding in
> > > >
> > > >
> > >
> >
> [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > > > SLF4J: Found binding in
> > > >
> > > >
> > >
> >
> [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for
> > an
> > > > explanation.
> > > > SLF4J: Actual binding is of type
> > [org.slf4j.impl.Log4jLoggerFactory]
> > > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > > 106.131
> > > > sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
> > > > org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed:
> > 106.131
> > > > sec <<< ERROR!
> > > > java.util.concurrent.TimeoutException: Futures timed out after
> > > [100000
> > > > milliseconds]
> > > > at
> > > > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> > > > at
> > > >
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> > > > at
> > > > scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> > > > at
> > > >
> > > >
> > >
> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> > > > at scala.concurrent.Await$.result(package.scala:107)
> > > > at akka.remote.Remoting.start(Remoting.scala:180)
> > > > at
> > > >
> > akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
> > > > at
> > > akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
> > > > at
> > > > akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
> > > > at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
> > > > at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
> > > > at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
> > > > at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
> > > > at akka.actor.ActorSystem$.create(ActorSystem.scala:67)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:82)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:105)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.<init>(FlinkMiniCluster.scala:61)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.<init>(LocalFlinkMiniCluster.scala:44)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.startFlinkMiniCluster(FlinkInterpreter.java:297)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.initializeFlinkEnv(FlinkInterpreter.java:152)
> > > > at
> > > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:115)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:40)
> > > >
> > > >
> > > > Results :
> > > >
> > > > Tests in error:
> > > > FlinkInterpreterTest.setUp:40 » Timeout Futures timed out
> after
> > > > [100000 millis...
> > > > ```
> > > >
> > > >
> > > > ---
> > > > If your project is set up for it, you can reply to this email and
> have
> > > your
> > > > reply appear on GitHub as well. If your project does not have this
> > > feature
> > > > enabled and wishes so, or if the feature is enabled but not working,
> > > please
> > > > contact infrastructure at infrastructure@apache.org or file a JIRA
> > > ticket
> > > > with INFRA.
> > > > ---
> > > >
> > >
> >
>
Re: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter
for Apach...
Posted by Till Rohrmann <tr...@apache.org>.
Hmm that is interesting. I always thought that in the case of multiple
SLF4J bindings, slf4j will simply pick one of them (first, last, random).
On Thu, May 21, 2015 at 12:34 PM, Stephan Ewen <se...@apache.org> wrote:
> The actor system fails to start. Here are the suspicious log lines:
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/home/travis/.m2/
> repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-
> 1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/home/travis/build/
> apache/incubator-zeppelin/zeppelin-interpreter/target/
> lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
>
> Seems that SLF4J has more than one binding. I remember that Akka crashed
> because of that before.
>
> As a simple fix, can you try and exclude the SLF4J jar from your build
> somehow? Or set it to "provided" in the Flink POM?
>
>
>
>
>
> On Thu, May 21, 2015 at 11:49 AM, Till Rohrmann <tr...@apache.org>
> wrote:
>
> > I'll try to reproduce this problem locally on my machine.
> >
> > On Thu, May 21, 2015 at 11:25 AM, Fabian Hueske <fh...@gmail.com>
> wrote:
> >
> > > Hi Flink folks,
> > >
> > > the Flink interpreter PR for Apache Zeppelin is blocked by a failing
> test
> > > case (see below).
> > > Does anybody have an idea what is going on and can maybe help to
> resolve
> > > the problem?
> > >
> > > Thanks, Fabian
> > >
> > > ---------- Forwarded message ----------
> > > From: Leemoonsoo <gi...@git.apache.org>
> > > Date: 2015-05-21 6:35 GMT+02:00
> > > Subject: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44
> > Interpreter
> > > for Apach...
> > > To: dev@zeppelin.incubator.apache.org
> > >
> > >
> > > Github user Leemoonsoo commented on the pull request:
> > >
> > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104132384
> > >
> > > This PR has some test using FlinkMiniCluster.
> > > Tests are fine when i run them on my IDE.
> > > But if i run them using maven, it fails. (CI is failing, too)
> > >
> > > I think everything is ready, except for this test failure. Any
> advice
> > > is very appreciated.
> > > here's failure message
> > >
> > > ```
> > > -------------------------------------------------------
> > > T E S T S
> > > -------------------------------------------------------
> > > Running org.apache.zeppelin.flink.FlinkInterpreterTest
> > > SLF4J: Class path contains multiple SLF4J bindings.
> > > SLF4J: Found binding in
> > >
> > >
> >
> [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > > SLF4J: Found binding in
> > >
> > >
> >
> [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for
> an
> > > explanation.
> > > SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory]
> > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 106.131
> > > sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
> > > org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed:
> 106.131
> > > sec <<< ERROR!
> > > java.util.concurrent.TimeoutException: Futures timed out after
> > [100000
> > > milliseconds]
> > > at
> > > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> > > at
> > > scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> > > at
> > > scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> > > at
> > >
> > >
> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> > > at scala.concurrent.Await$.result(package.scala:107)
> > > at akka.remote.Remoting.start(Remoting.scala:180)
> > > at
> > >
> akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
> > > at
> > akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
> > > at
> > > akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
> > > at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
> > > at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
> > > at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
> > > at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
> > > at akka.actor.ActorSystem$.create(ActorSystem.scala:67)
> > > at
> > >
> > >
> >
> org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:82)
> > > at
> > >
> > >
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:105)
> > > at
> > >
> > >
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.<init>(FlinkMiniCluster.scala:61)
> > > at
> > >
> > >
> >
> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.<init>(LocalFlinkMiniCluster.scala:44)
> > > at
> > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.startFlinkMiniCluster(FlinkInterpreter.java:297)
> > > at
> > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.initializeFlinkEnv(FlinkInterpreter.java:152)
> > > at
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:115)
> > > at
> > >
> > >
> >
> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:40)
> > >
> > >
> > > Results :
> > >
> > > Tests in error:
> > > FlinkInterpreterTest.setUp:40 » Timeout Futures timed out after
> > > [100000 millis...
> > > ```
> > >
> > >
> > > ---
> > > If your project is set up for it, you can reply to this email and have
> > your
> > > reply appear on GitHub as well. If your project does not have this
> > feature
> > > enabled and wishes so, or if the feature is enabled but not working,
> > please
> > > contact infrastructure at infrastructure@apache.org or file a JIRA
> > ticket
> > > with INFRA.
> > > ---
> > >
> >
>
Re: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter
for Apach...
Posted by Stephan Ewen <se...@apache.org>.
The actor system fails to start. Here are the suspicious log lines:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/travis/.m2/
repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-
1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/travis/build/
apache/incubator-zeppelin/zeppelin-interpreter/target/
lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Seems that SLF4J has more than one binding. I remember that Akka crashed
because of that before.
As a simple fix, can you try and exclude the SLF4J jar from your build
somehow? Or set it to "provided" in the Flink POM?
On Thu, May 21, 2015 at 11:49 AM, Till Rohrmann <tr...@apache.org>
wrote:
> I'll try to reproduce this problem locally on my machine.
>
> On Thu, May 21, 2015 at 11:25 AM, Fabian Hueske <fh...@gmail.com> wrote:
>
> > Hi Flink folks,
> >
> > the Flink interpreter PR for Apache Zeppelin is blocked by a failing test
> > case (see below).
> > Does anybody have an idea what is going on and can maybe help to resolve
> > the problem?
> >
> > Thanks, Fabian
> >
> > ---------- Forwarded message ----------
> > From: Leemoonsoo <gi...@git.apache.org>
> > Date: 2015-05-21 6:35 GMT+02:00
> > Subject: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44
> Interpreter
> > for Apach...
> > To: dev@zeppelin.incubator.apache.org
> >
> >
> > Github user Leemoonsoo commented on the pull request:
> >
> >
> >
> https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104132384
> >
> > This PR has some test using FlinkMiniCluster.
> > Tests are fine when i run them on my IDE.
> > But if i run them using maven, it fails. (CI is failing, too)
> >
> > I think everything is ready, except for this test failure. Any advice
> > is very appreciated.
> > here's failure message
> >
> > ```
> > -------------------------------------------------------
> > T E S T S
> > -------------------------------------------------------
> > Running org.apache.zeppelin.flink.FlinkInterpreterTest
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> >
> >
> [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> >
> >
> [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> > explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 106.131
> > sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
> > org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 106.131
> > sec <<< ERROR!
> > java.util.concurrent.TimeoutException: Futures timed out after
> [100000
> > milliseconds]
> > at
> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> > at
> > scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> > at
> > scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> > at
> >
> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> > at scala.concurrent.Await$.result(package.scala:107)
> > at akka.remote.Remoting.start(Remoting.scala:180)
> > at
> > akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
> > at
> akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
> > at
> > akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
> > at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
> > at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
> > at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
> > at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
> > at akka.actor.ActorSystem$.create(ActorSystem.scala:67)
> > at
> >
> >
> org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:82)
> > at
> >
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:105)
> > at
> >
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.<init>(FlinkMiniCluster.scala:61)
> > at
> >
> >
> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.<init>(LocalFlinkMiniCluster.scala:44)
> > at
> >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.startFlinkMiniCluster(FlinkInterpreter.java:297)
> > at
> >
> >
> org.apache.zeppelin.flink.FlinkInterpreter.initializeFlinkEnv(FlinkInterpreter.java:152)
> > at
> >
> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:115)
> > at
> >
> >
> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:40)
> >
> >
> > Results :
> >
> > Tests in error:
> > FlinkInterpreterTest.setUp:40 » Timeout Futures timed out after
> > [100000 millis...
> > ```
> >
> >
> > ---
> > If your project is set up for it, you can reply to this email and have
> your
> > reply appear on GitHub as well. If your project does not have this
> feature
> > enabled and wishes so, or if the feature is enabled but not working,
> please
> > contact infrastructure at infrastructure@apache.org or file a JIRA
> ticket
> > with INFRA.
> > ---
> >
>
Re: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter
for Apach...
Posted by Till Rohrmann <tr...@apache.org>.
I'll try to reproduce this problem locally on my machine.
On Thu, May 21, 2015 at 11:25 AM, Fabian Hueske <fh...@gmail.com> wrote:
> Hi Flink folks,
>
> the Flink interpreter PR for Apache Zeppelin is blocked by a failing test
> case (see below).
> Does anybody have an idea what is going on and can maybe help to resolve
> the problem?
>
> Thanks, Fabian
>
> ---------- Forwarded message ----------
> From: Leemoonsoo <gi...@git.apache.org>
> Date: 2015-05-21 6:35 GMT+02:00
> Subject: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter
> for Apach...
> To: dev@zeppelin.incubator.apache.org
>
>
> Github user Leemoonsoo commented on the pull request:
>
>
> https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104132384
>
> This PR has some test using FlinkMiniCluster.
> Tests are fine when i run them on my IDE.
> But if i run them using maven, it fails. (CI is failing, too)
>
> I think everything is ready, except for this test failure. Any advice
> is very appreciated.
> here's failure message
>
> ```
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.zeppelin.flink.FlinkInterpreterTest
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
> [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
>
> [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 106.131
> sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
> org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 106.131
> sec <<< ERROR!
> java.util.concurrent.TimeoutException: Futures timed out after [100000
> milliseconds]
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> at
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> at
> scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
> at
>
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> at scala.concurrent.Await$.result(package.scala:107)
> at akka.remote.Remoting.start(Remoting.scala:180)
> at
> akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
> at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
> at
> akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
> at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
> at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
> at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
> at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
> at akka.actor.ActorSystem$.create(ActorSystem.scala:67)
> at
>
> org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:82)
> at
>
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:105)
> at
>
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.<init>(FlinkMiniCluster.scala:61)
> at
>
> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.<init>(LocalFlinkMiniCluster.scala:44)
> at
>
> org.apache.zeppelin.flink.FlinkInterpreter.startFlinkMiniCluster(FlinkInterpreter.java:297)
> at
>
> org.apache.zeppelin.flink.FlinkInterpreter.initializeFlinkEnv(FlinkInterpreter.java:152)
> at
> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:115)
> at
>
> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:40)
>
>
> Results :
>
> Tests in error:
> FlinkInterpreterTest.setUp:40 » Timeout Futures timed out after
> [100000 millis...
> ```
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
Fwd: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44
Interpreter for Apach...
Posted by Fabian Hueske <fh...@gmail.com>.
Hi Flink folks,
the Flink interpreter PR for Apache Zeppelin is blocked by a failing test
case (see below).
Does anybody have an idea what is going on and can maybe help to resolve
the problem?
Thanks, Fabian
---------- Forwarded message ----------
From: Leemoonsoo <gi...@git.apache.org>
Date: 2015-05-21 6:35 GMT+02:00
Subject: [GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter
for Apach...
To: dev@zeppelin.incubator.apache.org
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104132384
This PR has some test using FlinkMiniCluster.
Tests are fine when i run them on my IDE.
But if i run them using maven, it fails. (CI is failing, too)
I think everything is ready, except for this test failure. Any advice
is very appreciated.
here's failure message
```
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.zeppelin.flink.FlinkInterpreterTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 106.131
sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 106.131
sec <<< ERROR!
java.util.concurrent.TimeoutException: Futures timed out after [100000
milliseconds]
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at akka.remote.Remoting.start(Remoting.scala:180)
at
akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
at
akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
at akka.actor.ActorSystem$.create(ActorSystem.scala:67)
at
org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:82)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:105)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.<init>(FlinkMiniCluster.scala:61)
at
org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.<init>(LocalFlinkMiniCluster.scala:44)
at
org.apache.zeppelin.flink.FlinkInterpreter.startFlinkMiniCluster(FlinkInterpreter.java:297)
at
org.apache.zeppelin.flink.FlinkInterpreter.initializeFlinkEnv(FlinkInterpreter.java:152)
at
org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:115)
at
org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:40)
Results :
Tests in error:
FlinkInterpreterTest.setUp:40 » Timeout Futures timed out after
[100000 millis...
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104132384
This PR has some test using FlinkMiniCluster.
Tests are fine when i run them on my IDE.
But if i run them using maven, it fails. (CI is failing, too)
I think everything is ready, except for this test failure. Any advice is very appreciated.
here's failure message
```
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.zeppelin.flink.FlinkInterpreterTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 106.131 sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
org.apache.zeppelin.flink.FlinkInterpreterTest Time elapsed: 106.131 sec <<< ERROR!
java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at akka.remote.Remoting.start(Remoting.scala:180)
at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:619)
at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:616)
at akka.actor.ActorSystemImpl._start(ActorSystem.scala:616)
at akka.actor.ActorSystemImpl.start(ActorSystem.scala:633)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:142)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:119)
at akka.actor.ActorSystem$.create(ActorSystem.scala:67)
at org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:82)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:105)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.<init>(FlinkMiniCluster.scala:61)
at org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.<init>(LocalFlinkMiniCluster.scala:44)
at org.apache.zeppelin.flink.FlinkInterpreter.startFlinkMiniCluster(FlinkInterpreter.java:297)
at org.apache.zeppelin.flink.FlinkInterpreter.initializeFlinkEnv(FlinkInterpreter.java:152)
at org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:115)
at org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:40)
Results :
Tests in error:
FlinkInterpreterTest.setUp:40 » Timeout Futures timed out after [100000 millis...
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-102975295
I'm very excited to see Flink support in Zeppelin :+1:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-102982009
@Leemoonsoo Great!!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104473572
Really appreciate for investigating the problem.
I think this is wrong in Zeppelin side. That, Zeppelin used to have dependency configuration for spark in pom.xml not in spark/pom.xml. I think it should be moved into spark/pom.xml to not interfere the other modules.
I'll create issue for it in Zeppelin side.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-104313952
I figured out the problem: It is a wrong protobuf-java version. `flink-runtime` depends on `akka-remote_2.10:2.3.7` which depends on `protobuf-java:2.5.0`. However, in the Zeppelin parent pom the `protobuf-java` version is manually set to `2.4.1`. There is a comment which declares this as a work-around to make Zeppelin build with Hadoop/YARN 2.2.
As a quick work-around you can set the protobuf-java version to 2.5.0 in the zeppelin-flink `pom.xml`. I'll also open a JIRA to shade the protobuf and akka dependency away in the flink project. This should solve this issue permanently.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-44 Interpreter for Apach...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/75#issuecomment-102967250
@jongyoul Yes it works!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---