You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "G.Long" <jd...@gmail.com> on 2012/04/27 16:31:46 UTC
Does Solr fit my needs?
Hi there :)
I'm looking for a way to save xml files into some sort of database and
i'm wondering if Solr would fit my needs.
The xml files I want to save have a lot of child nodes which also
contain child nodes with multiple values. The depth level can be more
than 10.
After having indexed the files, I would like to be able to query for
subparts of those xml files and be able to reconstruct them as xml files
with all their children included. However, I'm wondering if it is
possible with an index like solr lucene to keep or easily recover the
structure of my xml data?
Thanks for your help,
Regards,
Gary
Re: Does Solr fit my needs?
Posted by Shashi Kant <sk...@sloan.mit.edu>.
We have used both Solr and graph databases for our XML file indexing. Both
are equivalent in terms of performance, but a graph db (such as Neo4j)
offers a lot more flexibility in joining across the nodes and traversing.
If your data is strictly hierarchical Solr might do it, alternately suggest
looking at a Graph database such as Neo4j.
On Fri, Apr 27, 2012 at 10:36 AM, Bob Sandiford <
bob.sandiford@sirsidynix.com> wrote:
> ndexing and searching of the specific fields, it is certainly possible to
> retrieve the xml file. While Solr isn't a DB, it does allow a binary field
> to be associated with an index document. We store a GZipped XML file in a
> binary field and retrieve that under certain conditions to get at original
> document information. We've found that Solr can handle these much faster
> than our DB can do. (We regularly index a large portion of our documents,
> and the XML files are prone to frequent changes). If you DO keep such a
> blob in your Solr index, make sure you retrieve that field ONLY when you
> really want it...
RE: Does Solr fit my needs?
Posted by Bob Sandiford <bo...@sirsidynix.com>.
Without speaking directly to the indexing and searching of the specific fields, it is certainly possible to retrieve the xml file. While Solr isn't a DB, it does allow a binary field to be associated with an index document. We store a GZipped XML file in a binary field and retrieve that under certain conditions to get at original document information. We've found that Solr can handle these much faster than our DB can do. (We regularly index a large portion of our documents, and the XML files are prone to frequent changes). If you DO keep such a blob in your Solr index, make sure you retrieve that field ONLY when you really want it...
Now - if your XML files are relatively static (i.e. only change rarely, or only have new ones) then it still might make sense to use a real DB to store those, and just keep the primary key to the DB row in the Solr index.
Bob Sandiford | Lead Software Engineer | SirsiDynix
P: 800.288.8020 X6943 | Bob.Sandiford@sirsidynix.com
www.sirsidynix.com
Register for the 2012 COSUGI User Group Conference today for early bird pricing!
May 2-5 at Disney's Coronado Springs Resort - Lake Buena Vista, Florida
Join the conversation: Like us on Facebook! Follow us on Twitter!
-----Original Message-----
From: G.Long [mailto:jdevgl@gmail.com]
Sent: Friday, April 27, 2012 10:32 AM
To: solr-user@lucene.apache.org
Subject: Does Solr fit my needs?
Hi there :)
I'm looking for a way to save xml files into some sort of database and i'm wondering if Solr would fit my needs.
The xml files I want to save have a lot of child nodes which also contain child nodes with multiple values. The depth level can be more than 10.
After having indexed the files, I would like to be able to query for subparts of those xml files and be able to reconstruct them as xml files with all their children included. However, I'm wondering if it is possible with an index like solr lucene to keep or easily recover the structure of my xml data?
Thanks for your help,
Regards,
Gary
Re: Does Solr fit my needs?
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
no problem - you are welcome.
Nothing out-of-the-box yet. Only approach is ready
http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
https://issues.apache.org/jira/browse/SOLR-3076
Regards
On Mon, Apr 30, 2012 at 12:06 PM, G.Long <jd...@gmail.com> wrote:
> Hi :)
>
> Thank you all for your answers. I'll try these solutions :)
>
> Kind regards,
>
> Gary
>
> Le 27/04/2012 16:31, G.Long a écrit :
>
>> Hi there :)
>>
>> I'm looking for a way to save xml files into some sort of database and
>> i'm wondering if Solr would fit my needs.
>> The xml files I want to save have a lot of child nodes which also contain
>> child nodes with multiple values. The depth level can be more than 10.
>>
>> After having indexed the files, I would like to be able to query for
>> subparts of those xml files and be able to reconstruct them as xml files
>> with all their children included. However, I'm wondering if it is possible
>> with an index like solr lucene to keep or easily recover the structure of
>> my xml data?
>>
>> Thanks for your help,
>>
>> Regards,
>>
>> Gary
>>
>
>
--
Sincerely yours
Mikhail Khludnev
Tech Lead
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Does Solr fit my needs?
Posted by "G.Long" <jd...@gmail.com>.
Hi :)
Thank you all for your answers. I'll try these solutions :)
Kind regards,
Gary
Le 27/04/2012 16:31, G.Long a écrit :
> Hi there :)
>
> I'm looking for a way to save xml files into some sort of database and
> i'm wondering if Solr would fit my needs.
> The xml files I want to save have a lot of child nodes which also
> contain child nodes with multiple values. The depth level can be more
> than 10.
>
> After having indexed the files, I would like to be able to query for
> subparts of those xml files and be able to reconstruct them as xml
> files with all their children included. However, I'm wondering if it
> is possible with an index like solr lucene to keep or easily recover
> the structure of my xml data?
>
> Thanks for your help,
>
> Regards,
>
> Gary
Re: Does Solr fit my needs?
Posted by Bill Bell <bi...@gmail.com>.
You could use SQL Server and External Fields in Solr to get what you need from the database on result of the query.
Bill Bell
Sent from mobile
On Apr 27, 2012, at 8:31 AM, "G.Long" <jd...@gmail.com> wrote:
> Hi there :)
>
> I'm looking for a way to save xml files into some sort of database and i'm wondering if Solr would fit my needs.
> The xml files I want to save have a lot of child nodes which also contain child nodes with multiple values. The depth level can be more than 10.
>
> After having indexed the files, I would like to be able to query for subparts of those xml files and be able to reconstruct them as xml files with all their children included. However, I'm wondering if it is possible with an index like solr lucene to keep or easily recover the structure of my xml data?
>
> Thanks for your help,
>
> Regards,
>
> Gary
Re: Does Solr fit my needs?
Posted by Walter Underwood <wu...@wunderwood.org>.
Solr will not keep the structure of your XML data. Solr and Lucene have a flat data model. You can map hierarchy into that, but it can be a lot of work.
I recommend starting with a dedicated XML database. MarkLogic is commercial, but they have added a free developer license that can be used for some production (see limitations here: http://community.marklogic.com/express).
I would say to use "an XML database", but there really is no other production-ready XML database. eXist is nice, but it can be 100X slower than MarkLogic.
I used to work at MarkLogic and I've implemented Solr at two major web companies. If you really want to work in XML, you'll be a lot happier with MarkLogic.
wunder
On Apr 27, 2012, at 7:31 AM, G.Long wrote:
> Hi there :)
>
> I'm looking for a way to save xml files into some sort of database and i'm wondering if Solr would fit my needs.
> The xml files I want to save have a lot of child nodes which also contain child nodes with multiple values. The depth level can be more than 10.
>
> After having indexed the files, I would like to be able to query for subparts of those xml files and be able to reconstruct them as xml files with all their children included. However, I'm wondering if it is possible with an index like solr lucene to keep or easily recover the structure of my xml data?
>
> Thanks for your help,
>
> Regards,
>
> Gary