You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Alex McLintock <al...@gmail.com> on 2009/07/02 14:32:45 UTC
RDF Data Store on Hadoop
I'm looking to build up data for an RDF Data store using Hadoop.
I could just generate lots of RDF XML files - or a big one - and feed
it into Apache Jena..
However it seems to me that it would be best if I used a hadoop aware
distributed triplestore so that my data stayed on the data nodes.
I see that there is the Heart project ( http://rdf-proj.blogspot.com/
) but it doesnt seem very active.
Does anyone have any recommendations for a usable RDF data store which
I can use with Hadoop? Or should I consider this outside of the Hadoop
world and just put it on one machine? Is Heart "nearly there" and just
needs a helping hand?
I am tempted to bypass the RDF triplestore and role my own using hBase
but I dont want to re-invent the wheel.
Cheers
Alex
Re: RDF Data Store on Hadoop
Posted by Youen Péron <yo...@univ-ubs.fr>.
Hi
> I see that there is the Heart project ( http://rdf-proj.blogspot.com/
> ) but it doesnt seem very active.
The trac is active (3 days) :
http://heart.korea.ac.kr/trac/browser
and the the mailing list too :
http://groups.google.com/group/hrdfstore
but this project is still in development
Youen
RE: RDF Data Store on Hadoop
Posted by Brian MacKay <Br...@MEDecision.com>.
Hi Alex,
I have on several occasions, had the same consideration as you.
"Build a Hadoop aware distributed triplestore"
And I came to the same conclusions. "Heart doesn't seem very active" or
very developed.
I don't know of any other open source triple stores using Hadoop, so
that leaves the option of developing without re-inventing the wheel.
What you may want to consider is something I have been considering.
Build a virtual triple store using Hive over Hadoop in a manner similar
to D2RQ over relational databases.
http://www4.wiwiss.fu-berlin.de/bizer/D2RQ/
http://hadoop.apache.org/hive/
If you or anyone else is interested developing this idea into open a
source project, please contact me.
Thanks,
Brian
brianwmackay@msn.com
-----Original Message-----
From: Alex McLintock [mailto:alex.mclintock@gmail.com]
Sent: Thursday, July 02, 2009 8:33 AM
To: common-user@hadoop.apache.org
Subject: RDF Data Store on Hadoop
I'm looking to build up data for an RDF Data store using Hadoop.
I could just generate lots of RDF XML files - or a big one - and feed
it into Apache Jena..
However it seems to me that it would be best if I used a hadoop aware
distributed triplestore so that my data stayed on the data nodes.
I see that there is the Heart project ( http://rdf-proj.blogspot.com/
) but it doesnt seem very active.
Does anyone have any recommendations for a usable RDF data store which
I can use with Hadoop? Or should I consider this outside of the Hadoop
world and just put it on one machine? Is Heart "nearly there" and just
needs a helping hand?
I am tempted to bypass the RDF triplestore and role my own using hBase
but I dont want to re-invent the wheel.
Cheers
Alex
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this message in error, please contact the sender and delete the material
from any computer.
Re: RDF Data Store on Hadoop
Posted by Hyunsik Choi <c0...@gmail.com>.
Hello,
I'm a member of heart project. As you see, it is in development. We
faced some challenges to be solved for efficient and scalable RDF
store. Actually, we have already experienced some fails, but we
continuously try to that. Not all archives are published in the heart
blog or trac.
Actually, it is easy to store RDF into Hbase in spite of modifying
Hbase, and some kinds of queries are implemented easily. Like you, we
think mapping predicates to columns in column families, and it is
inspired by the paper 'Scalable Semantic Web Data Management Using
Vertical Partitioning'. As you know, however, column family can not be
altered or added into Hbase table without disabling Hbase table. It
restricts to many features.
Besides, to the best of my knowledge, I think that M/R is
inappropriate to RDF data or graph data. In Hbase, a RDF data set
(sorted in their subject) is partitioned into many tablets that
resides across many region servers. When a group graph pattern query
is issued, each mapper has to retrieve subgraphs matched to the given
query. But, each node will yield many candidates partially matched to
the given query because a RDF data set is disseminated across a number
of region servers. As a results, In reduce step, candidates from one
region server need to be joined to other candidates from other region
servers. However, assigning intermediate data (i.e., candidates) to
appropriate reducers are very difficult. Of course, assigning only one
reducer is very straightforward way, but it apparent to cause
scalability problem.
Therefore, It is a very important issue, and we try to build new
programming model for graph data
(http://wiki.apache.org/hadoop/Hamburg).
--
Hyunsik Choi
Database & Information Systems Group,
Korea University
On Fri, Jul 3, 2009 at 3:29 AM, Amandeep Khurana<am...@gmail.com> wrote:
> I can share the data model right here with you. Beyond the data model, the
> MR jobs etc are specific to the data sources I am pulling into Hbase to
> connect with each other.
>
> I've attached an image that represents the table structure basics.
>
> Essentially, the column family and column identifier used in combination
> represent the predicate. The row id is the subject and the cell value is the
> object..
>
>
>
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Thu, Jul 2, 2009 at 11:20 AM, Brian MacKay <Br...@medecision.com>
> wrote:
>>
>> I understand, if you would consider open sourcing it as a rough
>> prototype, let us know. ?
>>
>> -----Original Message-----
>> From: Amandeep Khurana [mailto:amansk@gmail.com]
>> Sent: Thursday, July 02, 2009 2:15 PM
>> To: common-user@hadoop.apache.org
>> Subject: Re: RDF Data Store on Hadoop
>>
>> Its a prototype right now and in a nascent stage. I havent made it open.
>> Essentially, its storing triples in Hbase so its just the data model
>> thats
>> solving some problems that I was working on. I dont have a SPARQL engine
>> built over it yet.
>>
>>
>>
>>
>> Amandeep Khurana
>> Computer Science Graduate Student
>> University of California, Santa Cruz
>>
>>
>> On Thu, Jul 2, 2009 at 11:13 AM, Brian MacKay
>> <Br...@medecision.com>wrote:
>>
>> > Hi Amandeep,
>> >
>> > Is your custom RDF store over Hbase open source? If so, where is it
>> > hosted?
>> >
>> > Thanks,
>> > Brian
>> >
>> > -----Original Message-----
>> > From: Amandeep Khurana [mailto:amansk@gmail.com]
>> > Sent: Thursday, July 02, 2009 2:08 PM
>> > To: common-user@hadoop.apache.org
>> > Subject: Re: RDF Data Store on Hadoop
>> >
>> > Hi
>> >
>> > I have been working on this as well. There are a couple of more
>> threads
>> > on
>> > this and the Hbase mailing list about this. One is pretty recent - its
>> > about
>> > graph algorithms using map reduce.
>> >
>> > I built a custom RDF store over Hbase. Its not really an RDF store by
>> > all
>> > means, but the data model is something that can be extended over to
>> > support
>> > all specifications of RDF.
>> >
>> > The Heart project hasnt been active for some time now. If we have
>> people
>> > interested, we can take it on and work on creating an RDF store over
>> > Hbase
>> > alongwith graph algorithms using MR.
>> >
>> > Amandeep
>> >
>> >
>> > Amandeep Khurana
>> > Computer Science Graduate Student
>> > University of California, Santa Cruz
>> >
>> >
>> > On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock
>> > <al...@gmail.com>wrote:
>> >
>> > > I'm looking to build up data for an RDF Data store using Hadoop.
>> > > I could just generate lots of RDF XML files - or a big one - and
>> feed
>> > > it into Apache Jena..
>> > > However it seems to me that it would be best if I used a hadoop
>> aware
>> > > distributed triplestore so that my data stayed on the data nodes.
>> > >
>> > > I see that there is the Heart project (
>> http://rdf-proj.blogspot.com/
>> > > ) but it doesnt seem very active.
>> > >
>> > > Does anyone have any recommendations for a usable RDF data store
>> which
>> > > I can use with Hadoop? Or should I consider this outside of the
>> Hadoop
>> > > world and just put it on one machine? Is Heart "nearly there" and
>> just
>> > > needs a helping hand?
>> > >
>> > > I am tempted to bypass the RDF triplestore and role my own using
>> hBase
>> > > but I dont want to re-invent the wheel.
>> > >
>> > > Cheers
>> > >
>> > > Alex
>> > >
>> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>> _ _ _
>> >
>> > The information transmitted is intended only for the person or entity
>> to
>> > which it is addressed and may contain confidential and/or privileged
>> > material. Any review, retransmission, dissemination or other use of,
>> or
>> > taking of any action in reliance upon, this information by persons or
>> > entities other than the intended recipient is prohibited. If you
>> received
>> > this message in error, please contact the sender and delete the
>> material
>> > from any computer.
>> >
>> >
>> >
>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>> _
>>
>> The information transmitted is intended only for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. Any review, retransmission, dissemination or other use of, or
>> taking of any action in reliance upon, this information by persons or
>> entities other than the intended recipient is prohibited. If you received
>> this message in error, please contact the sender and delete the material
>> from any computer.
>>
>>
>
>
Re: RDF Data Store on Hadoop
Posted by Yuan-Fang Li <li...@gmail.com>.
I recently came across this OSS project called BigData [1], with the
following description from their web site:
Bigdata(R) comes packaged with a very high-performance RDF store supporting
RDF(S) and OWL Lite inference. The Bigdata RDF Store is currently the only
RDF database capable of operating distributed on a cluster. The Bigdata RDF
Store was designed specifically to meet requirements for very large scale
semantic alignment and federation. RDF is a Semantic Web technology
particularly well-suited to modeling graph-shaped data and metadata, such as
an associative entity-link model, whereby actors are linked to one another
in an ad-hoc fashion within the context of an evolving ontology of concepts
for entity types and link types related to a particular problem domain. The
Bigdata RDF Store is used operationally in data harvesting systems to create
mash-ups of structured, semi-structured, and unstructured data from myriad
sources in a schema-flexible manner.
Having had a very brief look at the API, it seems that BigData is
interfacing with Sesame for RDF-related processing. It also seems that this
project has demonstrated some great scalability.
Does anybody have any experience with BigData?
Best wishes
Yuan-Fang
1. http://www.systap.com/bigdata.htm
On Fri, Jul 3, 2009 at 4:29 AM, Amandeep Khurana <am...@gmail.com> wrote:
> I can share the data model right here with you. Beyond the data model, the
> MR jobs etc are specific to the data sources I am pulling into Hbase to
> connect with each other.
>
> I've attached an image that represents the table structure basics.
>
> Essentially, the column family and column identifier used in combination
> represent the predicate. The row id is the subject and the cell value is the
> object..
>
>
>
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Thu, Jul 2, 2009 at 11:20 AM, Brian MacKay <Brian.MacKay@medecision.com
> > wrote:
>
>> I understand, if you would consider open sourcing it as a rough
>> prototype, let us know. ?
>>
>> -----Original Message-----
>> From: Amandeep Khurana [mailto:amansk@gmail.com]
>> Sent: Thursday, July 02, 2009 2:15 PM
>> To: common-user@hadoop.apache.org
>> Subject: Re: RDF Data Store on Hadoop
>>
>> Its a prototype right now and in a nascent stage. I havent made it open.
>> Essentially, its storing triples in Hbase so its just the data model
>> thats
>> solving some problems that I was working on. I dont have a SPARQL engine
>> built over it yet.
>>
>>
>>
>>
>> Amandeep Khurana
>> Computer Science Graduate Student
>> University of California, Santa Cruz
>>
>>
>> On Thu, Jul 2, 2009 at 11:13 AM, Brian MacKay
>> <Br...@medecision.com>wrote:
>>
>> > Hi Amandeep,
>> >
>> > Is your custom RDF store over Hbase open source? If so, where is it
>> > hosted?
>> >
>> > Thanks,
>> > Brian
>> >
>> > -----Original Message-----
>> > From: Amandeep Khurana [mailto:amansk@gmail.com]
>> > Sent: Thursday, July 02, 2009 2:08 PM
>> > To: common-user@hadoop.apache.org
>> > Subject: Re: RDF Data Store on Hadoop
>> >
>> > Hi
>> >
>> > I have been working on this as well. There are a couple of more
>> threads
>> > on
>> > this and the Hbase mailing list about this. One is pretty recent - its
>> > about
>> > graph algorithms using map reduce.
>> >
>> > I built a custom RDF store over Hbase. Its not really an RDF store by
>> > all
>> > means, but the data model is something that can be extended over to
>> > support
>> > all specifications of RDF.
>> >
>> > The Heart project hasnt been active for some time now. If we have
>> people
>> > interested, we can take it on and work on creating an RDF store over
>> > Hbase
>> > alongwith graph algorithms using MR.
>> >
>> > Amandeep
>> >
>> >
>> > Amandeep Khurana
>> > Computer Science Graduate Student
>> > University of California, Santa Cruz
>> >
>> >
>> > On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock
>> > <al...@gmail.com>wrote:
>> >
>> > > I'm looking to build up data for an RDF Data store using Hadoop.
>> > > I could just generate lots of RDF XML files - or a big one - and
>> feed
>> > > it into Apache Jena..
>> > > However it seems to me that it would be best if I used a hadoop
>> aware
>> > > distributed triplestore so that my data stayed on the data nodes.
>> > >
>> > > I see that there is the Heart project (
>> http://rdf-proj.blogspot.com/
>> > > ) but it doesnt seem very active.
>> > >
>> > > Does anyone have any recommendations for a usable RDF data store
>> which
>> > > I can use with Hadoop? Or should I consider this outside of the
>> Hadoop
>> > > world and just put it on one machine? Is Heart "nearly there" and
>> just
>> > > needs a helping hand?
>> > >
>> > > I am tempted to bypass the RDF triplestore and role my own using
>> hBase
>> > > but I dont want to re-invent the wheel.
>> > >
>> > > Cheers
>> > >
>> > > Alex
>> > >
>> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>> _ _ _
>> >
>> > The information transmitted is intended only for the person or entity
>> to
>> > which it is addressed and may contain confidential and/or privileged
>> > material. Any review, retransmission, dissemination or other use of,
>> or
>> > taking of any action in reliance upon, this information by persons or
>> > entities other than the intended recipient is prohibited. If you
>> received
>> > this message in error, please contact the sender and delete the
>> material
>> > from any computer.
>> >
>> >
>> >
>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>> _
>>
>> The information transmitted is intended only for the person or entity to
>> which it is addressed and may contain confidential and/or privileged
>> material. Any review, retransmission, dissemination or other use of, or
>> taking of any action in reliance upon, this information by persons or
>> entities other than the intended recipient is prohibited. If you received
>> this message in error, please contact the sender and delete the material
>> from any computer.
>>
>>
>>
>
Re: RDF Data Store on Hadoop
Posted by Amandeep Khurana <am...@gmail.com>.
I can share the data model right here with you. Beyond the data model, the
MR jobs etc are specific to the data sources I am pulling into Hbase to
connect with each other.
I've attached an image that represents the table structure basics.
Essentially, the column family and column identifier used in combination
represent the predicate. The row id is the subject and the cell value is the
object..
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Jul 2, 2009 at 11:20 AM, Brian MacKay
<Br...@medecision.com>wrote:
> I understand, if you would consider open sourcing it as a rough
> prototype, let us know. ?
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Thursday, July 02, 2009 2:15 PM
> To: common-user@hadoop.apache.org
> Subject: Re: RDF Data Store on Hadoop
>
> Its a prototype right now and in a nascent stage. I havent made it open.
> Essentially, its storing triples in Hbase so its just the data model
> thats
> solving some problems that I was working on. I dont have a SPARQL engine
> built over it yet.
>
>
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Thu, Jul 2, 2009 at 11:13 AM, Brian MacKay
> <Br...@medecision.com>wrote:
>
> > Hi Amandeep,
> >
> > Is your custom RDF store over Hbase open source? If so, where is it
> > hosted?
> >
> > Thanks,
> > Brian
> >
> > -----Original Message-----
> > From: Amandeep Khurana [mailto:amansk@gmail.com]
> > Sent: Thursday, July 02, 2009 2:08 PM
> > To: common-user@hadoop.apache.org
> > Subject: Re: RDF Data Store on Hadoop
> >
> > Hi
> >
> > I have been working on this as well. There are a couple of more
> threads
> > on
> > this and the Hbase mailing list about this. One is pretty recent - its
> > about
> > graph algorithms using map reduce.
> >
> > I built a custom RDF store over Hbase. Its not really an RDF store by
> > all
> > means, but the data model is something that can be extended over to
> > support
> > all specifications of RDF.
> >
> > The Heart project hasnt been active for some time now. If we have
> people
> > interested, we can take it on and work on creating an RDF store over
> > Hbase
> > alongwith graph algorithms using MR.
> >
> > Amandeep
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
> >
> > On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock
> > <al...@gmail.com>wrote:
> >
> > > I'm looking to build up data for an RDF Data store using Hadoop.
> > > I could just generate lots of RDF XML files - or a big one - and
> feed
> > > it into Apache Jena..
> > > However it seems to me that it would be best if I used a hadoop
> aware
> > > distributed triplestore so that my data stayed on the data nodes.
> > >
> > > I see that there is the Heart project (
> http://rdf-proj.blogspot.com/
> > > ) but it doesnt seem very active.
> > >
> > > Does anyone have any recommendations for a usable RDF data store
> which
> > > I can use with Hadoop? Or should I consider this outside of the
> Hadoop
> > > world and just put it on one machine? Is Heart "nearly there" and
> just
> > > needs a helping hand?
> > >
> > > I am tempted to bypass the RDF triplestore and role my own using
> hBase
> > > but I dont want to re-invent the wheel.
> > >
> > > Cheers
> > >
> > > Alex
> > >
> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _
> >
> > The information transmitted is intended only for the person or entity
> to
> > which it is addressed and may contain confidential and/or privileged
> > material. Any review, retransmission, dissemination or other use of,
> or
> > taking of any action in reliance upon, this information by persons or
> > entities other than the intended recipient is prohibited. If you
> received
> > this message in error, please contact the sender and delete the
> material
> > from any computer.
> >
> >
> >
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipient is prohibited. If you received
> this message in error, please contact the sender and delete the material
> from any computer.
>
>
>
RE: RDF Data Store on Hadoop
Posted by Brian MacKay <Br...@MEDecision.com>.
I understand, if you would consider open sourcing it as a rough
prototype, let us know. ?
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Thursday, July 02, 2009 2:15 PM
To: common-user@hadoop.apache.org
Subject: Re: RDF Data Store on Hadoop
Its a prototype right now and in a nascent stage. I havent made it open.
Essentially, its storing triples in Hbase so its just the data model
thats
solving some problems that I was working on. I dont have a SPARQL engine
built over it yet.
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Jul 2, 2009 at 11:13 AM, Brian MacKay
<Br...@medecision.com>wrote:
> Hi Amandeep,
>
> Is your custom RDF store over Hbase open source? If so, where is it
> hosted?
>
> Thanks,
> Brian
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Thursday, July 02, 2009 2:08 PM
> To: common-user@hadoop.apache.org
> Subject: Re: RDF Data Store on Hadoop
>
> Hi
>
> I have been working on this as well. There are a couple of more
threads
> on
> this and the Hbase mailing list about this. One is pretty recent - its
> about
> graph algorithms using map reduce.
>
> I built a custom RDF store over Hbase. Its not really an RDF store by
> all
> means, but the data model is something that can be extended over to
> support
> all specifications of RDF.
>
> The Heart project hasnt been active for some time now. If we have
people
> interested, we can take it on and work on creating an RDF store over
> Hbase
> alongwith graph algorithms using MR.
>
> Amandeep
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock
> <al...@gmail.com>wrote:
>
> > I'm looking to build up data for an RDF Data store using Hadoop.
> > I could just generate lots of RDF XML files - or a big one - and
feed
> > it into Apache Jena..
> > However it seems to me that it would be best if I used a hadoop
aware
> > distributed triplestore so that my data stayed on the data nodes.
> >
> > I see that there is the Heart project (
http://rdf-proj.blogspot.com/
> > ) but it doesnt seem very active.
> >
> > Does anyone have any recommendations for a usable RDF data store
which
> > I can use with Hadoop? Or should I consider this outside of the
Hadoop
> > world and just put it on one machine? Is Heart "nearly there" and
just
> > needs a helping hand?
> >
> > I am tempted to bypass the RDF triplestore and role my own using
hBase
> > but I dont want to re-invent the wheel.
> >
> > Cheers
> >
> > Alex
> >
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _
>
> The information transmitted is intended only for the person or entity
to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of,
or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipient is prohibited. If you
received
> this message in error, please contact the sender and delete the
material
> from any computer.
>
>
>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this message in error, please contact the sender and delete the material
from any computer.
Re: RDF Data Store on Hadoop
Posted by Amandeep Khurana <am...@gmail.com>.
Its a prototype right now and in a nascent stage. I havent made it open.
Essentially, its storing triples in Hbase so its just the data model thats
solving some problems that I was working on. I dont have a SPARQL engine
built over it yet.
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Jul 2, 2009 at 11:13 AM, Brian MacKay
<Br...@medecision.com>wrote:
> Hi Amandeep,
>
> Is your custom RDF store over Hbase open source? If so, where is it
> hosted?
>
> Thanks,
> Brian
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Thursday, July 02, 2009 2:08 PM
> To: common-user@hadoop.apache.org
> Subject: Re: RDF Data Store on Hadoop
>
> Hi
>
> I have been working on this as well. There are a couple of more threads
> on
> this and the Hbase mailing list about this. One is pretty recent - its
> about
> graph algorithms using map reduce.
>
> I built a custom RDF store over Hbase. Its not really an RDF store by
> all
> means, but the data model is something that can be extended over to
> support
> all specifications of RDF.
>
> The Heart project hasnt been active for some time now. If we have people
> interested, we can take it on and work on creating an RDF store over
> Hbase
> alongwith graph algorithms using MR.
>
> Amandeep
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock
> <al...@gmail.com>wrote:
>
> > I'm looking to build up data for an RDF Data store using Hadoop.
> > I could just generate lots of RDF XML files - or a big one - and feed
> > it into Apache Jena..
> > However it seems to me that it would be best if I used a hadoop aware
> > distributed triplestore so that my data stayed on the data nodes.
> >
> > I see that there is the Heart project ( http://rdf-proj.blogspot.com/
> > ) but it doesnt seem very active.
> >
> > Does anyone have any recommendations for a usable RDF data store which
> > I can use with Hadoop? Or should I consider this outside of the Hadoop
> > world and just put it on one machine? Is Heart "nearly there" and just
> > needs a helping hand?
> >
> > I am tempted to bypass the RDF triplestore and role my own using hBase
> > but I dont want to re-invent the wheel.
> >
> > Cheers
> >
> > Alex
> >
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipient is prohibited. If you received
> this message in error, please contact the sender and delete the material
> from any computer.
>
>
>
RE: RDF Data Store on Hadoop
Posted by Brian MacKay <Br...@MEDecision.com>.
Hi Amandeep,
Is your custom RDF store over Hbase open source? If so, where is it
hosted?
Thanks,
Brian
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Thursday, July 02, 2009 2:08 PM
To: common-user@hadoop.apache.org
Subject: Re: RDF Data Store on Hadoop
Hi
I have been working on this as well. There are a couple of more threads
on
this and the Hbase mailing list about this. One is pretty recent - its
about
graph algorithms using map reduce.
I built a custom RDF store over Hbase. Its not really an RDF store by
all
means, but the data model is something that can be extended over to
support
all specifications of RDF.
The Heart project hasnt been active for some time now. If we have people
interested, we can take it on and work on creating an RDF store over
Hbase
alongwith graph algorithms using MR.
Amandeep
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock
<al...@gmail.com>wrote:
> I'm looking to build up data for an RDF Data store using Hadoop.
> I could just generate lots of RDF XML files - or a big one - and feed
> it into Apache Jena..
> However it seems to me that it would be best if I used a hadoop aware
> distributed triplestore so that my data stayed on the data nodes.
>
> I see that there is the Heart project ( http://rdf-proj.blogspot.com/
> ) but it doesnt seem very active.
>
> Does anyone have any recommendations for a usable RDF data store which
> I can use with Hadoop? Or should I consider this outside of the Hadoop
> world and just put it on one machine? Is Heart "nearly there" and just
> needs a helping hand?
>
> I am tempted to bypass the RDF triplestore and role my own using hBase
> but I dont want to re-invent the wheel.
>
> Cheers
>
> Alex
>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you received
this message in error, please contact the sender and delete the material
from any computer.
Re: RDF Data Store on Hadoop
Posted by Amandeep Khurana <am...@gmail.com>.
Hi
I have been working on this as well. There are a couple of more threads on
this and the Hbase mailing list about this. One is pretty recent - its about
graph algorithms using map reduce.
I built a custom RDF store over Hbase. Its not really an RDF store by all
means, but the data model is something that can be extended over to support
all specifications of RDF.
The Heart project hasnt been active for some time now. If we have people
interested, we can take it on and work on creating an RDF store over Hbase
alongwith graph algorithms using MR.
Amandeep
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock <al...@gmail.com>wrote:
> I'm looking to build up data for an RDF Data store using Hadoop.
> I could just generate lots of RDF XML files - or a big one - and feed
> it into Apache Jena..
> However it seems to me that it would be best if I used a hadoop aware
> distributed triplestore so that my data stayed on the data nodes.
>
> I see that there is the Heart project ( http://rdf-proj.blogspot.com/
> ) but it doesnt seem very active.
>
> Does anyone have any recommendations for a usable RDF data store which
> I can use with Hadoop? Or should I consider this outside of the Hadoop
> world and just put it on one machine? Is Heart "nearly there" and just
> needs a helping hand?
>
> I am tempted to bypass the RDF triplestore and role my own using hBase
> but I dont want to re-invent the wheel.
>
> Cheers
>
> Alex
>
Re: RDF Data Store on Hadoop
Posted by Ted Dunning <te...@gmail.com>.
Heart looks related to Hama and I would expect a similar evolution.
On Thu, Jul 2, 2009 at 5:32 AM, Alex McLintock <al...@gmail.com>wrote:
> I see that there is the Heart project ( http://rdf-proj.blogspot.com/
> ) but it doesnt seem very active.
>