You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Michael Ravits <mi...@gmail.com> on 2015/05/17 23:53:02 UTC
Clojure Samza Task
Hi Everybody,
Trying to get up and running with a Clojure Samza Task.
I think that my question is Clojure specific but I guess that some of the
folks here are Clojure developers which might also know the answers.
I'm trying to run a job, using the same code as here:
https://github.com/apache/samza/blob/master/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala#L324
This is my clojure version:
(deftest a-test
(testing "debug"
(let [job ^StreamJob (->> (MapConfig. config)
(JobRunner.)
(.run))]
(do
(.waitForStatus job (ApplicationStatus/Running) 60000)))))
The same code in Java seems to start fine. But with Clojure I get the
following error. getClassLoader returns null in at
org.eclipse.jetty.util.log.Log$1.run (Log.java:94) which causes an NPE:
Caused by: java.lang.NullPointerException: null
at org.eclipse.jetty.util.log.Log$1.run (Log.java:94)
java.security.AccessController.doPrivileged (AccessController.java:-2)
org.eclipse.jetty.util.log.Log.<clinit> (Log.java:85)
org.eclipse.jetty.util.component.AbstractLifeCycle.<clinit>
(AbstractLifeCycle.java:33)
org.apache.samza.coordinator.server.HttpServer$.$lessinit$greater$default$4
(HttpServer.scala:65)
org.apache.samza.coordinator.JobCoordinator$.apply
(JobCoordinator.scala:56)
org.apache.samza.job.local.ThreadJobFactory.getJob
(ThreadJobFactory.scala:39)
org.apache.samza.job.JobRunner.run (JobRunner.scala:62)
assembler.core_test/fn (core_test.clj:47)
I'm a bit clueless right now as to how to investigation this further and
could not find any useful information on why this could happen.
Would be great if anyone could give me a clue as to what could be causing
this problem.
My second question is regarding this line:
val job = new JobRunner(new MapConfig(jobConfig)).run
How is that JobRunner.run() in Scala returns a Job object while in Java the
same method signature returns void?
Thanks,
Michael
Re: Clojure Samza Task
Posted by Michael Ravits <mi...@gmail.com>.
Hi Gustavo,
Thanks for answering!
I checked with a debugger both when running with Java and Clojure. In the
Clojure case Log.class.getClassLoader() returned null and caused the NPE.
I guess this is not the place to learn Scala but I'm still puzzled, how is
a function with a signature which means void actually returns something
else?
Sincerely,
Michael
On Mon, May 18, 2015 at 4:21 PM, Gustavo Anatoly <gu...@gmail.com>
wrote:
> Hi, Michael Ravits.
>
> The error happens when the Log class try to read the
> jetty-logging.properties file. Please take a look the line at (
> org.eclipse.jetty.util.log.Log.java:94)
>
> URL testProps = Log.class.getClassLoader().getResource(
> "jetty-logging.properties");
>
> Thus, the question is if this file exists.
>
> About the second question, the signature:
>
> class JobRunner(config: Config) extends Logging {
> def run() = {...}
>
> The operator = without any type represent a void return in Scala. Is
> the same when you declare something like this: def run(): Unit = {..}
>
> Unit documentation:
> http://www.scala-lang.org/api/current/index.html#scala.Unit
>
> Topic related at StackOverflow:
>
> http://stackoverflow.com/questions/9461880/should-i-use-unit-or-leave-out-the-return-type-for-my-scala-method
>
> Hope I helped you.
>
> Thanks.
>
>
> 2015-05-17 18:53 GMT-03:00 Michael Ravits <mi...@gmail.com>:
>
> > Hi Everybody,
> >
> > Trying to get up and running with a Clojure Samza Task.
> > I think that my question is Clojure specific but I guess that some of the
> > folks here are Clojure developers which might also know the answers.
> >
> > I'm trying to run a job, using the same code as here:
> >
> >
> https://github.com/apache/samza/blob/master/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala#L324
> >
> >
> > This is my clojure version:
> >
> > (deftest a-test
> > (testing "debug"
> > (let [job ^StreamJob (->> (MapConfig. config)
> > (JobRunner.)
> > (.run))]
> > (do
> > (.waitForStatus job (ApplicationStatus/Running) 60000)))))
> >
> >
> > The same code in Java seems to start fine. But with Clojure I get the
> > following error. getClassLoader returns null in at
> > org.eclipse.jetty.util.log.Log$1.run (Log.java:94) which causes an NPE:
> >
> > Caused by: java.lang.NullPointerException: null
> > at org.eclipse.jetty.util.log.Log$1.run (Log.java:94)
> > java.security.AccessController.doPrivileged
> (AccessController.java:-2)
> > org.eclipse.jetty.util.log.Log.<clinit> (Log.java:85)
> > org.eclipse.jetty.util.component.AbstractLifeCycle.<clinit>
> > (AbstractLifeCycle.java:33)
> >
> >
> org.apache.samza.coordinator.server.HttpServer$.$lessinit$greater$default$4
> > (HttpServer.scala:65)
> > org.apache.samza.coordinator.JobCoordinator$.apply
> > (JobCoordinator.scala:56)
> > org.apache.samza.job.local.ThreadJobFactory.getJob
> > (ThreadJobFactory.scala:39)
> > org.apache.samza.job.JobRunner.run (JobRunner.scala:62)
> > assembler.core_test/fn (core_test.clj:47)
> >
> >
> > I'm a bit clueless right now as to how to investigation this further and
> > could not find any useful information on why this could happen.
> > Would be great if anyone could give me a clue as to what could be causing
> > this problem.
> >
> > My second question is regarding this line:
> >
> > val job = new JobRunner(new MapConfig(jobConfig)).run
> >
> > How is that JobRunner.run() in Scala returns a Job object while in Java
> the
> > same method signature returns void?
> >
> > Thanks,
> > Michael
> >
>
Re: Clojure Samza Task
Posted by Michael Ravits <mi...@gmail.com>.
Hi Gustavo,
Thanks for answering!
I checked with a debugger both when running with Java and Clojure. In the
Clojure case Log.class.getClassLoader() returned null and caused the NPE.
I guess this is not the place to learn Scala but I'm still puzzled, how is
a function with a signature which means void actually returns something
else?
Sincerely,
Michael
On Mon, May 18, 2015 at 4:21 PM, Gustavo Anatoly <gu...@gmail.com>
wrote:
> Hi, Michael Ravits.
>
> The error happens when the Log class try to read the
> jetty-logging.properties file. Please take a look the line at (
> org.eclipse.jetty.util.log.Log.java:94)
>
> URL testProps = Log.class.getClassLoader().getResource(
> "jetty-logging.properties");
>
> Thus, the question is if this file exists.
>
> About the second question, the signature:
>
> class JobRunner(config: Config) extends Logging {
> def run() = {...}
>
> The operator = without any type represent a void return in Scala. Is
> the same when you declare something like this: def run(): Unit = {..}
>
> Unit documentation:
> http://www.scala-lang.org/api/current/index.html#scala.Unit
>
> Topic related at StackOverflow:
>
> http://stackoverflow.com/questions/9461880/should-i-use-unit-or-leave-out-the-return-type-for-my-scala-method
>
> Hope I helped you.
>
> Thanks.
>
>
> 2015-05-17 18:53 GMT-03:00 Michael Ravits <mi...@gmail.com>:
>
> > Hi Everybody,
> >
> > Trying to get up and running with a Clojure Samza Task.
> > I think that my question is Clojure specific but I guess that some of the
> > folks here are Clojure developers which might also know the answers.
> >
> > I'm trying to run a job, using the same code as here:
> >
> >
> https://github.com/apache/samza/blob/master/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala#L324
> >
> >
> > This is my clojure version:
> >
> > (deftest a-test
> > (testing "debug"
> > (let [job ^StreamJob (->> (MapConfig. config)
> > (JobRunner.)
> > (.run))]
> > (do
> > (.waitForStatus job (ApplicationStatus/Running) 60000)))))
> >
> >
> > The same code in Java seems to start fine. But with Clojure I get the
> > following error. getClassLoader returns null in at
> > org.eclipse.jetty.util.log.Log$1.run (Log.java:94) which causes an NPE:
> >
> > Caused by: java.lang.NullPointerException: null
> > at org.eclipse.jetty.util.log.Log$1.run (Log.java:94)
> > java.security.AccessController.doPrivileged
> (AccessController.java:-2)
> > org.eclipse.jetty.util.log.Log.<clinit> (Log.java:85)
> > org.eclipse.jetty.util.component.AbstractLifeCycle.<clinit>
> > (AbstractLifeCycle.java:33)
> >
> >
> org.apache.samza.coordinator.server.HttpServer$.$lessinit$greater$default$4
> > (HttpServer.scala:65)
> > org.apache.samza.coordinator.JobCoordinator$.apply
> > (JobCoordinator.scala:56)
> > org.apache.samza.job.local.ThreadJobFactory.getJob
> > (ThreadJobFactory.scala:39)
> > org.apache.samza.job.JobRunner.run (JobRunner.scala:62)
> > assembler.core_test/fn (core_test.clj:47)
> >
> >
> > I'm a bit clueless right now as to how to investigation this further and
> > could not find any useful information on why this could happen.
> > Would be great if anyone could give me a clue as to what could be causing
> > this problem.
> >
> > My second question is regarding this line:
> >
> > val job = new JobRunner(new MapConfig(jobConfig)).run
> >
> > How is that JobRunner.run() in Scala returns a Job object while in Java
> the
> > same method signature returns void?
> >
> > Thanks,
> > Michael
> >
>
Re: Clojure Samza Task
Posted by Gustavo Anatoly <gu...@gmail.com>.
Hi, Michael Ravits.
The error happens when the Log class try to read the
jetty-logging.properties file. Please take a look the line at (
org.eclipse.jetty.util.log.Log.java:94)
URL testProps = Log.class.getClassLoader().getResource(
"jetty-logging.properties");
Thus, the question is if this file exists.
About the second question, the signature:
class JobRunner(config: Config) extends Logging {
def run() = {...}
The operator = without any type represent a void return in Scala. Is
the same when you declare something like this: def run(): Unit = {..}
Unit documentation: http://www.scala-lang.org/api/current/index.html#scala.Unit
Topic related at StackOverflow:
http://stackoverflow.com/questions/9461880/should-i-use-unit-or-leave-out-the-return-type-for-my-scala-method
Hope I helped you.
Thanks.
2015-05-17 18:53 GMT-03:00 Michael Ravits <mi...@gmail.com>:
> Hi Everybody,
>
> Trying to get up and running with a Clojure Samza Task.
> I think that my question is Clojure specific but I guess that some of the
> folks here are Clojure developers which might also know the answers.
>
> I'm trying to run a job, using the same code as here:
>
> https://github.com/apache/samza/blob/master/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala#L324
>
>
> This is my clojure version:
>
> (deftest a-test
> (testing "debug"
> (let [job ^StreamJob (->> (MapConfig. config)
> (JobRunner.)
> (.run))]
> (do
> (.waitForStatus job (ApplicationStatus/Running) 60000)))))
>
>
> The same code in Java seems to start fine. But with Clojure I get the
> following error. getClassLoader returns null in at
> org.eclipse.jetty.util.log.Log$1.run (Log.java:94) which causes an NPE:
>
> Caused by: java.lang.NullPointerException: null
> at org.eclipse.jetty.util.log.Log$1.run (Log.java:94)
> java.security.AccessController.doPrivileged (AccessController.java:-2)
> org.eclipse.jetty.util.log.Log.<clinit> (Log.java:85)
> org.eclipse.jetty.util.component.AbstractLifeCycle.<clinit>
> (AbstractLifeCycle.java:33)
>
> org.apache.samza.coordinator.server.HttpServer$.$lessinit$greater$default$4
> (HttpServer.scala:65)
> org.apache.samza.coordinator.JobCoordinator$.apply
> (JobCoordinator.scala:56)
> org.apache.samza.job.local.ThreadJobFactory.getJob
> (ThreadJobFactory.scala:39)
> org.apache.samza.job.JobRunner.run (JobRunner.scala:62)
> assembler.core_test/fn (core_test.clj:47)
>
>
> I'm a bit clueless right now as to how to investigation this further and
> could not find any useful information on why this could happen.
> Would be great if anyone could give me a clue as to what could be causing
> this problem.
>
> My second question is regarding this line:
>
> val job = new JobRunner(new MapConfig(jobConfig)).run
>
> How is that JobRunner.run() in Scala returns a Job object while in Java the
> same method signature returns void?
>
> Thanks,
> Michael
>