You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by andrew_s <sh...@gmail.com> on 2011/04/04 11:31:52 UTC
Mongo REST interface and full data import
Hi everyone,
I'm trying to make a simple data import from MongoDB into Solr using REST
interface.
As an test example I've created schecma.xml like:
<?xml version="1.0" ?>
isbn
title
and data-import.xml as:
Unfortunately it's not working and I'm stuck on this place.
Could you please advise how correctly parser JSON format data?
Data format looks like:
{
"offset" : 0,
"rows": [
{ "_id" : { "$oid" : "4d9829412c8bd10644000000" }, "isbn" : "716739356",
"title" : "Proteins", "description" : "" } ,
{ "_id" : { "$oid" : "4d9829412c8bd10644000001" }, "isbn" :
"144433056X", "title" : "How to Assess Doctors and Health Professionals",
"description" : "" } ,
{ "_id" : { "$oid" : "4d9829412c8bd10644000002" }, "isbn" :
"1406208159", "title" : "Freestyle: Time Travel Guides: Pack B",
"description" : "Takes you on a trip through history to visit the great
ancient civilisations." } ,
"total_rows" : 3 ,
"query" : {} ,
"millis" : 0
}
Thank you.
--
View this message in context: http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2774479.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Mongo REST interface and full data import
Posted by andrew_s <sh...@gmail.com>.
Hi Stefan,
Thanks for clear explanation.
I've used XPathEntityProcessor as an example, because didn't found JSON
entity processor.
I'll write a script to generate XML file for data import.
Regards,
Andrew
--
View this message in context: http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2779208.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Mongo REST interface and full data import
Posted by Stefan Matheis <ma...@googlemail.com>.
andrew,
you're really wondering why the XPathEntityProcessor does not work
well, with a JSON-Structure !? The Links Erick posted are stating,
that you could push JSON-structured Data to a Solr-HTTP Interface ..
but not, that the DataImport Handler will work with them. IIRC there
is no way for processing JSON in DIH actually. Either expose XML-Data
and continue with DIH, or change the way and push JSON to the
/update/json interface
Regards
Stefan
On Tue, Apr 5, 2011 at 12:25 AM, andrew_s <sh...@gmail.com> wrote:
> Sorry for mistake with Solr version ... I'm using Solr 3.1
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2777319.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Mongo REST interface and full data import
Posted by andrew_s <sh...@gmail.com>.
Sorry for mistake with Solr version ... I'm using Solr 3.1
--
View this message in context: http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2777319.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Mongo REST interface and full data import
Posted by andrew_s <sh...@gmail.com>.
Hi Erick,
Thanks for your reply.
I'm using latest stable version (1.4).
About using JSON updates ... I have some experience how to setting up
data-import and delta-import from DB (it's pretty straight forward).
Not sure how it will work with updates from JSON.
Should I specify data-config.xml?
How delta import can be used for it?
BTW ... I've asked the same question on stackoverflow
http://stackoverflow.com/questions/5536770/mongo-rest-interface-and-full-data-import,
but no luck there. Anyway, it's possible to see data-config.xml and
schema.xml there.
Thanks for your help.
--
View this message in context: http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2776870.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Mongo REST interface and full data import
Posted by andrew_s <sh...@gmail.com>.
Thank you guys for your answers.
I didn't recognise that it will be so easy to do it and example from
http://wiki.apache.org/solr/UpdateJSON#Example works perfectly for me.
Regards,
Andrew
--
View this message in context: http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2808507.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Mongo REST interface and full data import
Posted by Erick Erickson <er...@gmail.com>.
I'm having trouble seeing your schema files, etc. I don't
know if gmail is stripping this on my end or whether
your e-mail is stripping it on upload, anyone else seeing this?
But to your question, what version are you using? From
Solr3.1 <http://wiki.apache.org/solr/Solr3.1> is the first version with JSON
support for updates.
See: http://wiki.apache.org/solr/UpdateJSON
<http://wiki.apache.org/solr/UpdateJSON>Best
Erick
On Mon, Apr 4, 2011 at 5:31 AM, andrew_s <sh...@gmail.com> wrote:
> Hi everyone,
>
> I'm trying to make a simple data import from MongoDB into Solr using REST
> interface.
>
> As an test example I've created schecma.xml like:
> <?xml version="1.0" ?>
>
>
>
>
>
>
>
>
>
>
>
>
>
> isbn
>
>
> title
>
>
>
>
>
>
> and data-import.xml as:
>
>
>
>
>
>
>
>
>
>
>
>
> Unfortunately it's not working and I'm stuck on this place.
>
> Could you please advise how correctly parser JSON format data?
>
>
> Data format looks like:
> {
> "offset" : 0,
> "rows": [
> { "_id" : { "$oid" : "4d9829412c8bd10644000000" }, "isbn" : "716739356",
> "title" : "Proteins", "description" : "" } ,
> { "_id" : { "$oid" : "4d9829412c8bd10644000001" }, "isbn" :
> "144433056X", "title" : "How to Assess Doctors and Health Professionals",
> "description" : "" } ,
> { "_id" : { "$oid" : "4d9829412c8bd10644000002" }, "isbn" :
> "1406208159", "title" : "Freestyle: Time Travel Guides: Pack B",
> "description" : "Takes you on a trip through history to visit the great
> ancient civilisations." } ,
> "total_rows" : 3 ,
> "query" : {} ,
> "millis" : 0
> }
>
>
> Thank you.
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Mongo-REST-interface-and-full-data-import-tp2774479p2774479.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>