Re: Advice on storage option

[Andy Seaborne <an...@epimorphics.com>]

On 28/03/11 19:49, McGibbney, Lewis John wrote:
> Hello list,
>
> I have been pondering over the following dilemma and would appreciate
> some guidance on best practice for implementation:
>
> I wish to store web data directly into a RDBMS then import this data
> to Solr for text based search functionality. I have a web-based
> ontology repository, these ontology's are used in a widget when a
> user submits a query to Solr. The reason I wish to initially store
> data in RDBMS is that I was planning on using either SquirrelRDF or
> D2R to send SPARQL queries and retrieve additional information from
> ontology's in this database E.g. data values, property values etc.
> The problem I have is that currently the ontology files are stored
> online and not in the RDBMS. It has become apparent that neither TDB
> or SDB are appropriate for storing as they cannot store 'normal' web
> documents (I hope this is correct), therefore can anyone suggest how
> I can overcome the problem of designing the system such that a
> suitable database can be populated with ontology files for SPARQL
> queries but also for data import to Solr for text based search.
>
> I hope I have explained clearly :0|   !!!

By a "normal web document" do you mean the ontology?  If so, then if 
it's RDF, or OWL recorded as RDF, it can be stored in SDB or TDB.

If you mean HTML page, one approach is to store a the bulk content in a 
store (file system, key-value store) and have a link to it (URI) in the 
RDF store.

	Andy

>
> Thank you Lewis
>
> Glasgow Caledonian University is a registered Scottish charity,
> number SC021474
>
> Winner: Times Higher Education’s Widening Participation Initiative of
> the Year 2009 and Herald Society’s Education Initiative of the Year
> 2009.
> http://www.gcu.ac.uk/newsevents/news/bycategory/theuniversity/1/name,6219,en.html
>
>  Winner: Times Higher Education’s Outstanding Support for Early
> Career Researchers of the Year 2010, GCU as a lead with Universities
> Scotland partners.
> http://www.gcu.ac.uk/newsevents/news/bycategory/theuniversity/1/name,15691,en.html

RE: Advice on storage option

["McGibbney, Lewis John" <Le...@gcu.ac.uk>]

Hi Andy thanks for reply

Well, my ambiguous reference to 'normal web document' was intended as per your second option... HTML pages. I can already see that there will be some design issues here!!!

My ontology's are OWL recorded as RDF. The part I was trying to make clear was that for simplicity in design of the system architecture, I was trying to mitigate against the requirement to diverge from RDBMS to a key-value store or other method you mentioned.

However having though about it, it is looking like I will store the ontology's in TDB/SDB then use ARQ instead of D2R. This will enable me to remove the RDBMS, leaving an independent Solr index, and fundamentally creating two specific information retrieval systems working within the framework.

To me this sounds like a more suitable and feasible option as at this stage I wish to keep simplicity at the core of the framework.

Does this make sense?
Thanks Lewis
--------------------------------------------------------------------------------------

By a "normal web document" do you mean the ontology?  If so, then if
it's RDF, or OWL recorded as RDF, it can be stored in SDB or TDB.

If you mean HTML page, one approach is to store a the bulk content in a
store (file system, key-value store) and have a link to it (URI) in the
RDF store.


Glasgow Caledonian University is a registered Scottish charity, number SC021474

Winner: Times Higher Education’s Widening Participation Initiative of the Year 2009 and Herald Society’s Education Initiative of the Year 2009.
http://www.gcu.ac.uk/newsevents/news/bycategory/theuniversity/1/name,6219,en.html

Winner: Times Higher Education’s Outstanding Support for Early Career Researchers of the Year 2010, GCU as a lead with Universities Scotland partners.
http://www.gcu.ac.uk/newsevents/news/bycategory/theuniversity/1/name,15691,en.html