You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Timothy Potter <th...@gmail.com> on 2016/07/21 16:28:31 UTC
Streaming expression - workers is zero somehow?
I'm working with 6.1.0 release and I have a single SolrCloud instance
with 1 shard / 1 replica. Somehow I'm triggering this, which from what
I can see, means workers == 0, but how? Shouldn't workers default to 1
I should mention that my streaming expression doesn't include any
workers, i.e. it is simply:
val hashJoinExpr =
s"""
| hashJoin(
| search(${ratingsCollection},
| q="*:*",
| fl="movie_id,user_id,rating",
| sort="movie_id asc",
| qt="/export",
| partitionKeys="movie_id"),
| hashed=search(${moviesCollection},
| q="*:*",
| fl="movie_id,title",
| sort="movie_id asc",
| qt="/export",
| partitionKeys="movie_id"),
| on="movie_id"
| )
""".stripMargin
2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase
- java.io.IOException: java.lang.RuntimeException:
java.lang.ArithmeticException: / by zero
at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752)
at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
at org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102)
at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91)
at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386)
at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064)
at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.lang.ArithmeticException: / by zero
at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
... 42 more
Caused by: java.lang.ArithmeticException: / by zero
at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
... 43 more
Re: Streaming expression - workers is zero somehow?
Posted by Joel Bernstein <jo...@gmail.com>.
Take a look at StatementImpl.constructStream()
This uses a SolrStream to connect to the /sql handler. You can use the same
approach to send a request to the /stream handler just by changing the
parameters. Then you can open and read the SolrStream.
We don't yet have a load balancing SolrStream.
Joel Bernstein
http://joelsolr.blogspot.com/
On Tue, Jul 26, 2016 at 11:27 AM, Timothy Potter <th...@gmail.com>
wrote:
> Ok, makes sense now, thanks Joel. We should probably add some earlier
> error checking vs. letting the code get all the way into the
> HashQParser.
>
> So this raises a separate question that I haven't been able to figure
> out, namely, do we have an example of just POST'ing the expression to
> the /stream handler from SolrJ client code? i.e. I don't want to parse
> and execute the streaming expression on the client side, rather, I
> want to post it to the server side. Currently, my client code is a big
> copy and paste of the /stream request handler (with 1 obvious omission
> ;-) Specifically, I wasn't able to figure out how to parse the tuple
> stream coming back using SolrJ code if I just post the expression to
> /stream.
>
> Tim
>
> On Tue, Jul 26, 2016 at 8:54 AM, Joel Bernstein <jo...@gmail.com>
> wrote:
> > The difference would be if you are compiling and running the expression
> in
> > a java class or sending it to the /stream handler to be compiled.
> >
> > If you're compiling it and running it locally you could get this error
> > because the StreamContext would not have the numWorkers variable set.
> >
> > The /stream handler always sets the numWorkers variable, so in theory you
> > would never see this error if the /stream handler is executing the
> > expression.
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> > On Tue, Jul 26, 2016 at 10:44 AM, Timothy Potter <th...@gmail.com>
> > wrote:
> >
> >> it's from a unit test, but not sure why that matters? If I wrap the
> >> expression in a parallel expression with explicit workers=1, then it
> >> works
> >>
> >> On Thu, Jul 21, 2016 at 11:13 AM, Joel Bernstein <jo...@gmail.com>
> >> wrote:
> >> > Are you getting this error from a test case you've setup or from a
> manual
> >> > call to the /stream handler?
> >> >
> >> > Joel Bernstein
> >> > http://joelsolr.blogspot.com/
> >> >
> >> > On Thu, Jul 21, 2016 at 12:28 PM, Timothy Potter <
> thelabdude@gmail.com>
> >> > wrote:
> >> >
> >> >> I'm working with 6.1.0 release and I have a single SolrCloud instance
> >> >> with 1 shard / 1 replica. Somehow I'm triggering this, which from
> what
> >> >> I can see, means workers == 0, but how? Shouldn't workers default to
> 1
> >> >>
> >> >> I should mention that my streaming expression doesn't include any
> >> >> workers, i.e. it is simply:
> >> >>
> >> >> val hashJoinExpr =
> >> >> s"""
> >> >> | hashJoin(
> >> >> | search(${ratingsCollection},
> >> >> | q="*:*",
> >> >> | fl="movie_id,user_id,rating",
> >> >> | sort="movie_id asc",
> >> >> | qt="/export",
> >> >> | partitionKeys="movie_id"),
> >> >> | hashed=search(${moviesCollection},
> >> >> | q="*:*",
> >> >> | fl="movie_id,title",
> >> >> | sort="movie_id asc",
> >> >> | qt="/export",
> >> >> | partitionKeys="movie_id"),
> >> >> | on="movie_id"
> >> >> | )
> >> >> """.stripMargin
> >> >>
> >> >>
> >> >>
> >> >> 2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase
> >> >> - java.io.IOException: java.lang.RuntimeException:
> >> >> java.lang.ArithmeticException: / by zero
> >> >> at
> >> >>
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
> >> >> at
> >> >>
> >>
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752)
> >> >> at
> >> >>
> >>
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735)
> >> >> at
> >> >> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102)
> >> >> at
> >> org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)
> >> >> at
> >> >>
> >>
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
> >> >> at
> >> >>
> >>
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)
> >> >> at
> >> >>
> >>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
> >> >> at
> >> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
> >> >> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
> >> >> at
> >> >>
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> >> >> at
> >> >>
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
> >> >> at
> >> >>
> >>
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> >> >> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> >> >> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> >> >> at
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
> >> >> at
> >> >>
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
> >> >> at java.lang.Thread.run(Thread.java:745)
> >> >> Caused by: java.lang.RuntimeException:
> java.lang.ArithmeticException: /
> >> by
> >> >> zero
> >> >> at
> >> >>
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
> >> >> at
> >> >>
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
> >> >> ... 42 more
> >> >> Caused by: java.lang.ArithmeticException: / by zero
> >> >> at
> >> >>
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
> >> >> ... 43 more
> >> >>
> >>
>
Re: Streaming expression - workers is zero somehow?
Posted by Timothy Potter <th...@gmail.com>.
Ok, makes sense now, thanks Joel. We should probably add some earlier
error checking vs. letting the code get all the way into the
HashQParser.
So this raises a separate question that I haven't been able to figure
out, namely, do we have an example of just POST'ing the expression to
the /stream handler from SolrJ client code? i.e. I don't want to parse
and execute the streaming expression on the client side, rather, I
want to post it to the server side. Currently, my client code is a big
copy and paste of the /stream request handler (with 1 obvious omission
;-) Specifically, I wasn't able to figure out how to parse the tuple
stream coming back using SolrJ code if I just post the expression to
/stream.
Tim
On Tue, Jul 26, 2016 at 8:54 AM, Joel Bernstein <jo...@gmail.com> wrote:
> The difference would be if you are compiling and running the expression in
> a java class or sending it to the /stream handler to be compiled.
>
> If you're compiling it and running it locally you could get this error
> because the StreamContext would not have the numWorkers variable set.
>
> The /stream handler always sets the numWorkers variable, so in theory you
> would never see this error if the /stream handler is executing the
> expression.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Tue, Jul 26, 2016 at 10:44 AM, Timothy Potter <th...@gmail.com>
> wrote:
>
>> it's from a unit test, but not sure why that matters? If I wrap the
>> expression in a parallel expression with explicit workers=1, then it
>> works
>>
>> On Thu, Jul 21, 2016 at 11:13 AM, Joel Bernstein <jo...@gmail.com>
>> wrote:
>> > Are you getting this error from a test case you've setup or from a manual
>> > call to the /stream handler?
>> >
>> > Joel Bernstein
>> > http://joelsolr.blogspot.com/
>> >
>> > On Thu, Jul 21, 2016 at 12:28 PM, Timothy Potter <th...@gmail.com>
>> > wrote:
>> >
>> >> I'm working with 6.1.0 release and I have a single SolrCloud instance
>> >> with 1 shard / 1 replica. Somehow I'm triggering this, which from what
>> >> I can see, means workers == 0, but how? Shouldn't workers default to 1
>> >>
>> >> I should mention that my streaming expression doesn't include any
>> >> workers, i.e. it is simply:
>> >>
>> >> val hashJoinExpr =
>> >> s"""
>> >> | hashJoin(
>> >> | search(${ratingsCollection},
>> >> | q="*:*",
>> >> | fl="movie_id,user_id,rating",
>> >> | sort="movie_id asc",
>> >> | qt="/export",
>> >> | partitionKeys="movie_id"),
>> >> | hashed=search(${moviesCollection},
>> >> | q="*:*",
>> >> | fl="movie_id,title",
>> >> | sort="movie_id asc",
>> >> | qt="/export",
>> >> | partitionKeys="movie_id"),
>> >> | on="movie_id"
>> >> | )
>> >> """.stripMargin
>> >>
>> >>
>> >>
>> >> 2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase
>> >> - java.io.IOException: java.lang.RuntimeException:
>> >> java.lang.ArithmeticException: / by zero
>> >> at
>> >>
>> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
>> >> at
>> >>
>> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752)
>> >> at
>> >>
>> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735)
>> >> at
>> >> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
>> >> at
>> >>
>> org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102)
>> >> at
>> org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91)
>> >> at
>> >>
>> org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386)
>> >> at
>> >>
>> org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064)
>> >> at
>> >>
>> org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234)
>> >> at
>> >>
>> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751)
>> >> at
>> >>
>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)
>> >> at
>> >>
>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)
>> >> at
>> >>
>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
>> >> at
>> >>
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)
>> >> at
>> >>
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
>> >> at
>> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
>> >> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
>> >> at
>> >>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
>> >> at
>> >>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
>> >> at
>> >>
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
>> >> at
>> >>
>> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
>> >> at
>> >>
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
>> >> at
>> >>
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
>> >> at
>> >>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
>> >> at
>> >>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>> >> at
>> >>
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
>> >> at
>> >>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>> >> at
>> >>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>> >> at
>> >>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> >> at
>> >>
>> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
>> >> at
>> >>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>> >> at org.eclipse.jetty.server.Server.handle(Server.java:518)
>> >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>> >> at
>> >>
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
>> >> at
>> >>
>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>> >> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>> >> at
>> >>
>> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>> >> at
>> >>
>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
>> >> at
>> >>
>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
>> >> at
>> >>
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
>> >> at
>> >>
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> Caused by: java.lang.RuntimeException: java.lang.ArithmeticException: /
>> by
>> >> zero
>> >> at
>> >>
>> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
>> >> at
>> >>
>> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
>> >> ... 42 more
>> >> Caused by: java.lang.ArithmeticException: / by zero
>> >> at
>> >>
>> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
>> >> ... 43 more
>> >>
>>
Re: Streaming expression - workers is zero somehow?
Posted by Joel Bernstein <jo...@gmail.com>.
The difference would be if you are compiling and running the expression in
a java class or sending it to the /stream handler to be compiled.
If you're compiling it and running it locally you could get this error
because the StreamContext would not have the numWorkers variable set.
The /stream handler always sets the numWorkers variable, so in theory you
would never see this error if the /stream handler is executing the
expression.
Joel Bernstein
http://joelsolr.blogspot.com/
On Tue, Jul 26, 2016 at 10:44 AM, Timothy Potter <th...@gmail.com>
wrote:
> it's from a unit test, but not sure why that matters? If I wrap the
> expression in a parallel expression with explicit workers=1, then it
> works
>
> On Thu, Jul 21, 2016 at 11:13 AM, Joel Bernstein <jo...@gmail.com>
> wrote:
> > Are you getting this error from a test case you've setup or from a manual
> > call to the /stream handler?
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> > On Thu, Jul 21, 2016 at 12:28 PM, Timothy Potter <th...@gmail.com>
> > wrote:
> >
> >> I'm working with 6.1.0 release and I have a single SolrCloud instance
> >> with 1 shard / 1 replica. Somehow I'm triggering this, which from what
> >> I can see, means workers == 0, but how? Shouldn't workers default to 1
> >>
> >> I should mention that my streaming expression doesn't include any
> >> workers, i.e. it is simply:
> >>
> >> val hashJoinExpr =
> >> s"""
> >> | hashJoin(
> >> | search(${ratingsCollection},
> >> | q="*:*",
> >> | fl="movie_id,user_id,rating",
> >> | sort="movie_id asc",
> >> | qt="/export",
> >> | partitionKeys="movie_id"),
> >> | hashed=search(${moviesCollection},
> >> | q="*:*",
> >> | fl="movie_id,title",
> >> | sort="movie_id asc",
> >> | qt="/export",
> >> | partitionKeys="movie_id"),
> >> | on="movie_id"
> >> | )
> >> """.stripMargin
> >>
> >>
> >>
> >> 2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase
> >> - java.io.IOException: java.lang.RuntimeException:
> >> java.lang.ArithmeticException: / by zero
> >> at
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
> >> at
> >>
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752)
> >> at
> >>
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735)
> >> at
> >> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
> >> at
> >>
> org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102)
> >> at
> org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91)
> >> at
> >>
> org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386)
> >> at
> >>
> org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064)
> >> at
> >>
> org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234)
> >> at
> >>
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751)
> >> at
> >>
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)
> >> at
> >>
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)
> >> at
> >>
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
> >> at
> >>
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)
> >> at
> >>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
> >> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
> >> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
> >> at
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> >> at
> >>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> >> at
> >>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
> >> at
> >>
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
> >> at
> >>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
> >> at
> >>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> >> at
> >>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
> >> at
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
> >> at
> >>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> >> at
> >>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> >> at
> >>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
> >> at
> >>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> >> at
> >>
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
> >> at
> >>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> >> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> >> at
> >>
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> >> at
> >>
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> >> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> >> at
> >>
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> >> at
> >>
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
> >> at
> >>
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
> >> at
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
> >> at
> >>
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: java.lang.RuntimeException: java.lang.ArithmeticException: /
> by
> >> zero
> >> at
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
> >> at
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
> >> ... 42 more
> >> Caused by: java.lang.ArithmeticException: / by zero
> >> at
> >>
> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
> >> ... 43 more
> >>
>
Re: Streaming expression - workers is zero somehow?
Posted by Timothy Potter <th...@gmail.com>.
it's from a unit test, but not sure why that matters? If I wrap the
expression in a parallel expression with explicit workers=1, then it
works
On Thu, Jul 21, 2016 at 11:13 AM, Joel Bernstein <jo...@gmail.com> wrote:
> Are you getting this error from a test case you've setup or from a manual
> call to the /stream handler?
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Thu, Jul 21, 2016 at 12:28 PM, Timothy Potter <th...@gmail.com>
> wrote:
>
>> I'm working with 6.1.0 release and I have a single SolrCloud instance
>> with 1 shard / 1 replica. Somehow I'm triggering this, which from what
>> I can see, means workers == 0, but how? Shouldn't workers default to 1
>>
>> I should mention that my streaming expression doesn't include any
>> workers, i.e. it is simply:
>>
>> val hashJoinExpr =
>> s"""
>> | hashJoin(
>> | search(${ratingsCollection},
>> | q="*:*",
>> | fl="movie_id,user_id,rating",
>> | sort="movie_id asc",
>> | qt="/export",
>> | partitionKeys="movie_id"),
>> | hashed=search(${moviesCollection},
>> | q="*:*",
>> | fl="movie_id,title",
>> | sort="movie_id asc",
>> | qt="/export",
>> | partitionKeys="movie_id"),
>> | on="movie_id"
>> | )
>> """.stripMargin
>>
>>
>>
>> 2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase
>> - java.io.IOException: java.lang.RuntimeException:
>> java.lang.ArithmeticException: / by zero
>> at
>> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
>> at
>> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752)
>> at
>> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
>> at
>> org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102)
>> at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)
>> at
>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)
>> at
>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
>> at
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
>> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
>> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
>> at
>> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> at
>> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>> at org.eclipse.jetty.server.Server.handle(Server.java:518)
>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>> at
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
>> at
>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>> at
>> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>> at
>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
>> at
>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.RuntimeException: java.lang.ArithmeticException: / by
>> zero
>> at
>> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
>> at
>> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
>> ... 42 more
>> Caused by: java.lang.ArithmeticException: / by zero
>> at
>> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
>> ... 43 more
>>
Re: Streaming expression - workers is zero somehow?
Posted by Joel Bernstein <jo...@gmail.com>.
Are you getting this error from a test case you've setup or from a manual
call to the /stream handler?
Joel Bernstein
http://joelsolr.blogspot.com/
On Thu, Jul 21, 2016 at 12:28 PM, Timothy Potter <th...@gmail.com>
wrote:
> I'm working with 6.1.0 release and I have a single SolrCloud instance
> with 1 shard / 1 replica. Somehow I'm triggering this, which from what
> I can see, means workers == 0, but how? Shouldn't workers default to 1
>
> I should mention that my streaming expression doesn't include any
> workers, i.e. it is simply:
>
> val hashJoinExpr =
> s"""
> | hashJoin(
> | search(${ratingsCollection},
> | q="*:*",
> | fl="movie_id,user_id,rating",
> | sort="movie_id asc",
> | qt="/export",
> | partitionKeys="movie_id"),
> | hashed=search(${moviesCollection},
> | q="*:*",
> | fl="movie_id,title",
> | sort="movie_id asc",
> | qt="/export",
> | partitionKeys="movie_id"),
> | on="movie_id"
> | )
> """.stripMargin
>
>
>
> 2016-07-21 10:08:44,596 [qtp2125832297-1073] ERROR RequestHandlerBase
> - java.io.IOException: java.lang.RuntimeException:
> java.lang.ArithmeticException: / by zero
> at
> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
> at
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:752)
> at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:735)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
> at
> org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:102)
> at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:91)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1386)
> at
> org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:1064)
> at
> org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1234)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1751)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1627)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:643)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
> at
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.ArithmeticException: / by
> zero
> at
> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
> at
> org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
> ... 42 more
> Caused by: java.lang.ArithmeticException: / by zero
> at
> org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
> ... 43 more
>