You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by ufuk yılmaz <uy...@vivaldi.net.INVALID> on 2021/01/21 08:16:34 UTC

Parallel streaming expression java.lang.IndexOutOfBoundsException

Hello all,

https://lucene.apache.org/solr/guide/8_4/stream-decorator-reference.html#parallel

I’m sending the same query in the docs, (just collection names changed) to my Solr but always getting the exception:

{
  "result-set":{
    "docs":[{
        "EXCEPTION":"java.lang.IndexOutOfBoundsException: Index 1 out of bounds for length 1",
        "EOF":true,
        "RESPONSE_TIME":93}]}}

My query:

null(
	parallel(
		WorkerCollection,
		rollup(
			search(
				colA,
				q="username: c*",
				qt="/export",
				fl="timestamp,user.id",
				sort="user.id asc",
				partitionKeys="user.id"
			),
			over="user.id",count(*)
		),
		workers="4",
		sort="timestamp asc"
	)
)

WorkerCollection has 1 shard and 4 replicas on 4 different machines. I double-triple checked for a silly syntax mistake but there’s none that I can see. I tried doing this a few months ago with no success, getting the same exception again. colA is on different machines from WorkerCollection. user.id is a string field, timestamp is long.

What am I missing?


Sent from Mail for Windows 10


RE: Parallel streaming expression java.lang.IndexOutOfBoundsException

Posted by ufuk yılmaz <uy...@vivaldi.net.INVALID>.
It only works when I set workers to 1, which defeats the point of parallel.

Sent from Mail for Windows 10

From: ufuk yılmaz
Sent: 21 January 2021 11:16
To: solr-user@lucene.apache.org
Subject: Parallel streaming expression java.lang.IndexOutOfBoundsException

Hello all,

https://lucene.apache.org/solr/guide/8_4/stream-decorator-reference.html#parallel

I’m sending the same query in the docs, (just collection names changed) to my Solr but always getting the exception:

{
  "result-set":{
    "docs":[{
        "EXCEPTION":"java.lang.IndexOutOfBoundsException: Index 1 out of bounds for length 1",
        "EOF":true,
        "RESPONSE_TIME":93}]}}

My query:

null(
	parallel(
		WorkerCollection,
		rollup(
			search(
				colA,
				q="username: c*",
				qt="/export",
				fl="timestamp,user.id",
				sort="user.id asc",
				partitionKeys="user.id"
			),
			over="user.id",count(*)
		),
		workers="4",
		sort="timestamp asc"
	)
)

WorkerCollection has 1 shard and 4 replicas on 4 different machines. I double-triple checked for a silly syntax mistake but there’s none that I can see. I tried doing this a few months ago with no success, getting the same exception again. colA is on different machines from WorkerCollection. user.id is a string field, timestamp is long.

What am I missing?


Sent from Mail for Windows 10



Re: Parallel streaming expression java.lang.IndexOutOfBoundsException

Posted by Joel Bernstein <jo...@gmail.com>.
Yes, this is not ideal. It means that the worker collection needs to have N
shards rather than N replicas. Changing this should not be difficult if
you'd like to provide a patch.


Joel Bernstein
http://joelsolr.blogspot.com/


On Thu, Jan 21, 2021 at 8:00 AM ufuk yılmaz <uy...@vivaldi.net.invalid>
wrote:

> Looked at the source code of the parallel stream and it seems I need equal
> number of SHARDS and workers count parameter. I thought I needed as many
> replicas, it was shards.
> Maybe helps someone.
>
> Sent from Mail for Windows 10
>
> From: ufuk yılmaz
> Sent: 21 January 2021 11:16
> To: solr-user@lucene.apache.org
> Subject: Parallel streaming expression java.lang.IndexOutOfBoundsException
>
> Hello all,
>
>
> https://lucene.apache.org/solr/guide/8_4/stream-decorator-reference.html#parallel
>
> I’m sending the same query in the docs, (just collection names changed) to
> my Solr but always getting the exception:
>
> {
>   "result-set":{
>     "docs":[{
>         "EXCEPTION":"java.lang.IndexOutOfBoundsException: Index 1 out of
> bounds for length 1",
>         "EOF":true,
>         "RESPONSE_TIME":93}]}}
>
> My query:
>
> null(
>         parallel(
>                 WorkerCollection,
>                 rollup(
>                         search(
>                                 colA,
>                                 q="username: c*",
>                                 qt="/export",
>                                 fl="timestamp,user.id",
>                                 sort="user.id asc",
>                                 partitionKeys="user.id"
>                         ),
>                         over="user.id",count(*)
>                 ),
>                 workers="4",
>                 sort="timestamp asc"
>         )
> )
>
> WorkerCollection has 1 shard and 4 replicas on 4 different machines. I
> double-triple checked for a silly syntax mistake but there’s none that I
> can see. I tried doing this a few months ago with no success, getting the
> same exception again. colA is on different machines from WorkerCollection.
> user.id is a string field, timestamp is long.
>
> What am I missing?
>
>
> Sent from Mail for Windows 10
>
>
>

RE: Parallel streaming expression java.lang.IndexOutOfBoundsException

Posted by ufuk yılmaz <uy...@vivaldi.net.INVALID>.
Looked at the source code of the parallel stream and it seems I need equal number of SHARDS and workers count parameter. I thought I needed as many replicas, it was shards.
Maybe helps someone.

Sent from Mail for Windows 10

From: ufuk yılmaz
Sent: 21 January 2021 11:16
To: solr-user@lucene.apache.org
Subject: Parallel streaming expression java.lang.IndexOutOfBoundsException

Hello all,

https://lucene.apache.org/solr/guide/8_4/stream-decorator-reference.html#parallel

I’m sending the same query in the docs, (just collection names changed) to my Solr but always getting the exception:

{
  "result-set":{
    "docs":[{
        "EXCEPTION":"java.lang.IndexOutOfBoundsException: Index 1 out of bounds for length 1",
        "EOF":true,
        "RESPONSE_TIME":93}]}}

My query:

null(
	parallel(
		WorkerCollection,
		rollup(
			search(
				colA,
				q="username: c*",
				qt="/export",
				fl="timestamp,user.id",
				sort="user.id asc",
				partitionKeys="user.id"
			),
			over="user.id",count(*)
		),
		workers="4",
		sort="timestamp asc"
	)
)

WorkerCollection has 1 shard and 4 replicas on 4 different machines. I double-triple checked for a silly syntax mistake but there’s none that I can see. I tried doing this a few months ago with no success, getting the same exception again. colA is on different machines from WorkerCollection. user.id is a string field, timestamp is long.

What am I missing?


Sent from Mail for Windows 10