You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Chetas Joshi <ch...@gmail.com> on 2016/10/20 21:49:26 UTC

/export handler to stream data using CloudSolrStream: JSONParse Exception

Hello,

I am using /export handler to stream data using CloudSolrStream.

I am using fl=uuid,space,timestamp where uuid and space are Strings and
timestamp is long. My query (q=.......) is not on these fields.

While reading the results from the Solr cloud, I get the following errors

org.noggit.JSONParser$ParseException: Expected ',' or '}':
char=5,position=110938
BEFORE='uuid":"0lG99s8vyaKB2I/I","space":"uuid","timestamp":1 5' AFTER='DB6
474294954},{"uuid":"0lG99sHT8P5e'


Or (For a different query


org.noggit.JSONParser$ParseException: Expected ',' or '}':
char=",position=122528
BEFORE=':1475618674},{"uuid":"Whz991tX6P4beuhp","space": 3076 "'
AFTER='uuid","timestamp":1476131442},{"uui'


Now what are the possible reasons of me getting this error?


Is this related to some kind of data corruption?


What are some of the things (possibly some characters in String) that JSON
will have hard time parsing?


The Solr version I use is 5.5.0


Thanks


Chetas.

Re: /export handler to stream data using CloudSolrStream: JSONParse Exception

Posted by Chetas Joshi <ch...@gmail.com>.

Just to the add to my previous question: I used dynamic shard splitting
while consuming data from the Solr collection using /export handler.

On Fri, Oct 21, 2016 at 2:27 PM, Chetas Joshi <ch...@gmail.com>
wrote:

> Thanks Joel.
>
> I will migrate to Solr 6.0.0.
>
> However, I have one more question. Have you come across any discussion
> about Spark-on-Solr corrupting the data?
>
> So, I am getting the JSONParse exceptions only for a collection on which I
> tried loading the data using Spark Dataframe API (which internally uses
> /export handler to stream data using CloudSolrStream).
>
> The data loading using CloudSolrStream API from all the other collections
> works fine.
>
> Just want to know if you have come across this issue.
>
> Thanks,
>
> Chetas.
>
>
>
> On Thu, Oct 20, 2016 at 7:03 PM, Joel Bernstein <jo...@gmail.com>
> wrote:
>
>> I suspect this is a bug with improperly escaped json. SOLR-7441
>> <https://issues.apache.org/jira/browse/SOLR-7441> resolved this issue and
>> released in Solr 6.0.
>>
>> There have been a large number of improvements, bug fixes, new features
>> and
>> much better error handling in Solr 6 Streaming Expressions.
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>> On Thu, Oct 20, 2016 at 5:49 PM, Chetas Joshi <ch...@gmail.com>
>> wrote:
>>
>> > Hello,
>> >
>> > I am using /export handler to stream data using CloudSolrStream.
>> >
>> > I am using fl=uuid,space,timestamp where uuid and space are Strings and
>> > timestamp is long. My query (q=.......) is not on these fields.
>> >
>> > While reading the results from the Solr cloud, I get the following
>> errors
>> >
>> > org.noggit.JSONParser$ParseException: Expected ',' or '}':
>> > char=5,position=110938
>> > BEFORE='uuid":"0lG99s8vyaKB2I/I","space":"uuid","timestamp":1 5'
>> > AFTER='DB6
>> > 474294954},{"uuid":"0lG99sHT8P5e'
>> >
>> >
>> > Or (For a different query
>> >
>> >
>> > org.noggit.JSONParser$ParseException: Expected ',' or '}':
>> > char=",position=122528
>> > BEFORE=':1475618674},{"uuid":"Whz991tX6P4beuhp","space": 3076 "'
>> > AFTER='uuid","timestamp":1476131442},{"uui'
>> >
>> >
>> > Now what are the possible reasons of me getting this error?
>> >
>> >
>> > Is this related to some kind of data corruption?
>> >
>> >
>> > What are some of the things (possibly some characters in String) that
>> JSON
>> > will have hard time parsing?
>> >
>> >
>> > The Solr version I use is 5.5.0
>> >
>> >
>> > Thanks
>> >
>> >
>> > Chetas.
>> >
>>
>
>

Re: /export handler to stream data using CloudSolrStream: JSONParse Exception

Posted by Chetas Joshi <ch...@gmail.com>.

Thanks Joel.

I will migrate to Solr 6.0.0.

However, I have one more question. Have you come across any discussion
about Spark-on-Solr corrupting the data?

So, I am getting the JSONParse exceptions only for a collection on which I
tried loading the data using Spark Dataframe API (which internally uses
/export handler to stream data using CloudSolrStream).

The data loading using CloudSolrStream API from all the other collections
works fine.

Just want to know if you have come across this issue.

Thanks,

Chetas.



On Thu, Oct 20, 2016 at 7:03 PM, Joel Bernstein <jo...@gmail.com> wrote:

> I suspect this is a bug with improperly escaped json. SOLR-7441
> <https://issues.apache.org/jira/browse/SOLR-7441> resolved this issue and
> released in Solr 6.0.
>
> There have been a large number of improvements, bug fixes, new features and
> much better error handling in Solr 6 Streaming Expressions.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Thu, Oct 20, 2016 at 5:49 PM, Chetas Joshi <ch...@gmail.com>
> wrote:
>
> > Hello,
> >
> > I am using /export handler to stream data using CloudSolrStream.
> >
> > I am using fl=uuid,space,timestamp where uuid and space are Strings and
> > timestamp is long. My query (q=.......) is not on these fields.
> >
> > While reading the results from the Solr cloud, I get the following errors
> >
> > org.noggit.JSONParser$ParseException: Expected ',' or '}':
> > char=5,position=110938
> > BEFORE='uuid":"0lG99s8vyaKB2I/I","space":"uuid","timestamp":1 5'
> > AFTER='DB6
> > 474294954},{"uuid":"0lG99sHT8P5e'
> >
> >
> > Or (For a different query
> >
> >
> > org.noggit.JSONParser$ParseException: Expected ',' or '}':
> > char=",position=122528
> > BEFORE=':1475618674},{"uuid":"Whz991tX6P4beuhp","space": 3076 "'
> > AFTER='uuid","timestamp":1476131442},{"uui'
> >
> >
> > Now what are the possible reasons of me getting this error?
> >
> >
> > Is this related to some kind of data corruption?
> >
> >
> > What are some of the things (possibly some characters in String) that
> JSON
> > will have hard time parsing?
> >
> >
> > The Solr version I use is 5.5.0
> >
> >
> > Thanks
> >
> >
> > Chetas.
> >
>

Re: /export handler to stream data using CloudSolrStream: JSONParse Exception

Posted by Joel Bernstein <jo...@gmail.com>.

I suspect this is a bug with improperly escaped json. SOLR-7441
<https://issues.apache.org/jira/browse/SOLR-7441> resolved this issue and
released in Solr 6.0.

There have been a large number of improvements, bug fixes, new features and
much better error handling in Solr 6 Streaming Expressions.

Joel Bernstein
http://joelsolr.blogspot.com/

On Thu, Oct 20, 2016 at 5:49 PM, Chetas Joshi <ch...@gmail.com>
wrote:

> Hello,
>
> I am using /export handler to stream data using CloudSolrStream.
>
> I am using fl=uuid,space,timestamp where uuid and space are Strings and
> timestamp is long. My query (q=.......) is not on these fields.
>
> While reading the results from the Solr cloud, I get the following errors
>
> org.noggit.JSONParser$ParseException: Expected ',' or '}':
> char=5,position=110938
> BEFORE='uuid":"0lG99s8vyaKB2I/I","space":"uuid","timestamp":1 5'
> AFTER='DB6
> 474294954},{"uuid":"0lG99sHT8P5e'
>
>
> Or (For a different query
>
>
> org.noggit.JSONParser$ParseException: Expected ',' or '}':
> char=",position=122528
> BEFORE=':1475618674},{"uuid":"Whz991tX6P4beuhp","space": 3076 "'
> AFTER='uuid","timestamp":1476131442},{"uui'
>
>
> Now what are the possible reasons of me getting this error?
>
>
> Is this related to some kind of data corruption?
>
>
> What are some of the things (possibly some characters in String) that JSON
> will have hard time parsing?
>
>
> The Solr version I use is 5.5.0
>
>
> Thanks
>
>
> Chetas.
>