You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Basmajian, Raffi" <rb...@oppenheimerfunds.com> on 2010/03/31 17:27:06 UTC
Using SPARQL against HBase
We are currently researching how to use SPARQL against data in Hbase. I
understand the use of Get and Scan classes in the Hbase API, but these
search classes do not return data in the same way SPARQL against RDF
data returns it. My colleagues and I were discussing that these types of
search results will require creating an in-memory graph first from
Hbase, then using SPARQL against that graph. We are not sure how this is
accomplished. Any advice would help, thank you
-RNY
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
I am looking for a solution to support "reading" data for now.
I believe the graph will contain data triples; not metadata.
We are looking at Hbase/Hadoop at the moment and not native graph
solutions.
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
Stack
Sent: Wednesday, March 31, 2010 11:51 AM
To: hbase-user@hadoop.apache.org
Subject: Re: Using SPARQL against HBase
Writes would update your in-memory graph and the backing hbase store?
The in-memory graph would hold all data or just metadata? You might
look at IHBase in the 'indexed' contrib to see how it loads an index on
region open (It subclasses hbase so it can catch key transistions)
Why HBase and not a native graph database?
Yours,
St.Ack
On Wed, Mar 31, 2010 at 8:27 AM, Basmajian, Raffi
<rb...@oppenheimerfunds.com> wrote:
> We are currently researching how to use SPARQL against data in Hbase.
> I understand the use of Get and Scan classes in the Hbase API, but
> these search classes do not return data in the same way SPARQL against
> RDF data returns it. My colleagues and I were discussing that these
> types of search results will require creating an in-memory graph first
> from Hbase, then using SPARQL against that graph. We are not sure how
> this is accomplished. Any advice would help, thank you
>
> -RNY
>
> ----------------------------------------------------------------------
> -------- This e-mail transmission may contain information that is
> proprietary, privileged and/or confidential and is intended
exclusively for the person(s) to whom it is addressed. Any use, copying,
retention or disclosure by any person other than the intended recipient
or the intended recipient's designees is strictly prohibited. If you are
not the intended recipient or their designee, please notify the sender
immediately by return e-mail and delete all copies. OppenheimerFunds
may, at its sole discretion, monitor, review, retain and/or disclose the
content of all email communications.
>
========================================================================
======
>
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
Re: Using SPARQL against HBase
Posted by Amandeep Khurana <am...@gmail.com>.
HBase has a simple API: get(), put() and scan() for the most part. There was
work done on a query language recently. Its called HBQL. Here's the link:
http://www.hbql.com/
-Amandeep
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 12:58 PM, Basmajian, Raffi <
rbasmajian@oppenheimerfunds.com> wrote:
> I am not looking to use SQL on Hbase, I want to know what is the
> counterpart to SQL for Hbase; if any.
>
> -----Original Message-----
> From: Michael Segel [mailto:michael_segel@hotmail.com]
> Sent: Wednesday, March 31, 2010 3:28 PM
> To: hbase-user@hadoop.apache.org
> Subject: RE: Using SPARQL against HBase
>
>
>
> Call me silly, but what is meant by a noSQL 'database' when people want
> to use SQL against it? :-)
>
> Ok, while it seems a bit silly, it is a serious question.
>
> -Mike
>
> > Date: Wed, 31 Mar 2010 11:56:51 -0700
> > From: apurtell@apache.org
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org
> >
> > Hi Raffi,
> >
> > To read up on fundamentals I suggest Google's BigTable paper:
> > http://labs.google.com/papers/bigtable.html
> >
> > Detail on how HBase implements the BigTable architecture within the
> Hadoop ecosystem can be found here:
> >
> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> >
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >
> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> > og.html
> >
> > Hope that helps,
> >
> > - Andy
> >
> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > > Subject: RE: Using SPARQL against HBase
> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > > SPARQL-like queries, then what type of query language can it respond
>
> > > to? In a traditional RDBMS database one would use SQL; so what is
> > > the counterpart query language with Hbase?
> >
> >
> >
> >
>
> _________________________________________________________________
> Hotmail: Trusted email with Microsoft's powerful SPAM protection.
> http://clk.atdmt.com/GBL/go/210850552/direct/01/
>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the person(s)
> to whom it is addressed. Any use, copying, retention or disclosure by any
> person other than the intended recipient or the intended recipient's
> designees is strictly prohibited. If you are not the intended recipient or
> their designee, please notify the sender immediately by return e-mail and
> delete all copies. OppenheimerFunds may, at its sole discretion, monitor,
> review, retain and/or disclose the content of all email communications.
>
> ==============================================================================
>
>
Re: RE: Using SPARQL against HBase
Posted by Ryan Rawson <ry...@gmail.com>.
Hbase has an API oriented interface, not a query language interface.
Natively the answer to your question is "none"
On Mar 31, 2010 12:59 PM, "Basmajian, Raffi" <
rbasmajian@oppenheimerfunds.com> wrote:
I am not looking to use SQL on Hbase, I want to know what is the
counterpart to SQL for Hbase; if any.
-----Original Message-----
From: Michael Segel [mailto:michael_segel@hotmail.com]
Sent: Wednesday,...
------------------------------------------------------------------------------
This e-mail transmiss...
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
I am not looking to use SQL on Hbase, I want to know what is the
counterpart to SQL for Hbase; if any.
-----Original Message-----
From: Michael Segel [mailto:michael_segel@hotmail.com]
Sent: Wednesday, March 31, 2010 3:28 PM
To: hbase-user@hadoop.apache.org
Subject: RE: Using SPARQL against HBase
Call me silly, but what is meant by a noSQL 'database' when people want
to use SQL against it? :-)
Ok, while it seems a bit silly, it is a serious question.
-Mike
> Date: Wed, 31 Mar 2010 11:56:51 -0700
> From: apurtell@apache.org
> Subject: RE: Using SPARQL against HBase
> To: hbase-user@hadoop.apache.org
>
> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper:
> http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the
Hadoop ecosystem can be found here:
>
> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
>
http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> og.html
>
> Hope that helps,
>
> - Andy
>
> > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > SPARQL-like queries, then what type of query language can it respond
> > to? In a traditional RDBMS database one would use SQL; so what is
> > the counterpart query language with Hbase?
>
>
>
>
_________________________________________________________________
Hotmail: Trusted email with Microsoft's powerful SPAM protection.
http://clk.atdmt.com/GBL/go/210850552/direct/01/
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
RE: Using SPARQL against HBase
Posted by Michael Segel <mi...@hotmail.com>.
Call me silly, but what is meant by a noSQL 'database' when people want to use SQL against it? :-)
Ok, while it seems a bit silly, it is a serious question.
-Mike
> Date: Wed, 31 Mar 2010 11:56:51 -0700
> From: apurtell@apache.org
> Subject: RE: Using SPARQL against HBase
> To: hbase-user@hadoop.apache.org
>
> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper: http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the Hadoop ecosystem can be found here:
>
> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
>
> Hope that helps,
>
> - Andy
>
> > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM
> > If Hbase can't respond to SPARQL-like queries, then what type
> > of query language can it respond to? In a traditional RDBMS
> > database one would use SQL; so what is the counterpart query
> > language with Hbase?
>
>
>
>
_________________________________________________________________
Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
http://clk.atdmt.com/GBL/go/210850552/direct/01/
Re: Using SPARQL against HBase
Posted by Andrew Purtell <ap...@apache.org>.
I expect Trend Micro could assign headcount if this were a serious project.
- Andy
> From: Vukasin Toroman <vu...@toroman.name>
> Subject: Re: Using SPARQL against HBase
> To: hbase-user@hadoop.apache.org
> Date: Wednesday, March 31, 2010, 12:52 PM
> Hi guys,
>
> I would be very interested in a collaboration on a project
> like that. Any plans on starting a project?
>
> Best,
> Vukasin
>
>
> On Mar 31, 2010, at 9:29 PM, Amandeep Khurana wrote:
>
> > Why do you need to build an in-memory graph which you
> would want to
> > read/write to? You could store the graph in HBase
> directly. As pointed out,
> > HBase might not be the best suited for SPARQL queries,
> but its not
> > impossible to do. Using the triples, you can form a
> graph that can be
> > represented in HBase as an adjacency list. I've stored
> graphs with 16-17M
> > nodes which was data equivalent to about 600M triples.
> And this was on a
> > small cluster and could certainly scale way more than
> 16M graph nodes.
> >
> > In case you are interested in working on SPARQL over
> HBase, we could
> > collaborate on it...
> >
> > -ak
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
> >
> > On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> <ap...@apache.org>wrote:
> >
> >> Hi Raffi,
> >>
> >> To read up on fundamentals I suggest Google's
> BigTable paper:
> >> http://labs.google.com/papers/bigtable.html
> >>
> >> Detail on how HBase implements the BigTable
> architecture within the Hadoop
> >> ecosystem can be found here:
> >>
> >> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> >> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >>
> >> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
> >>
> >> Hope that helps,
> >>
> >> - Andy
> >>
> >>> From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> >>> Subject: RE: Using SPARQL against HBase
> >>> To: hbase-user@hadoop.apache.org,
> apurtell@apache.org
> >>> Date: Wednesday, March 31, 2010, 11:42 AM
> >>> If Hbase can't respond to SPARQL-like queries,
> then what type
> >>> of query language can it respond to? In a
> traditional RDBMS
> >>> database one would use SQL; so what is the
> counterpart query
> >>> language with Hbase?
> >>
> >>
> >>
> >>
> >>
>
>
Re: Using SPARQL against HBase
Posted by Vukasin Toroman <vu...@toroman.name>.
Hi guys,
I would be very interested in a collaboration on a project like that. Any plans on starting a project?
Best,
Vukasin
On Mar 31, 2010, at 9:29 PM, Amandeep Khurana wrote:
> Why do you need to build an in-memory graph which you would want to
> read/write to? You could store the graph in HBase directly. As pointed out,
> HBase might not be the best suited for SPARQL queries, but its not
> impossible to do. Using the triples, you can form a graph that can be
> represented in HBase as an adjacency list. I've stored graphs with 16-17M
> nodes which was data equivalent to about 600M triples. And this was on a
> small cluster and could certainly scale way more than 16M graph nodes.
>
> In case you are interested in working on SPARQL over HBase, we could
> collaborate on it...
>
> -ak
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell <ap...@apache.org>wrote:
>
>> Hi Raffi,
>>
>> To read up on fundamentals I suggest Google's BigTable paper:
>> http://labs.google.com/papers/bigtable.html
>>
>> Detail on how HBase implements the BigTable architecture within the Hadoop
>> ecosystem can be found here:
>>
>> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
>> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>>
>> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
>>
>> Hope that helps,
>>
>> - Andy
>>
>>> From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
>>> Subject: RE: Using SPARQL against HBase
>>> To: hbase-user@hadoop.apache.org, apurtell@apache.org
>>> Date: Wednesday, March 31, 2010, 11:42 AM
>>> If Hbase can't respond to SPARQL-like queries, then what type
>>> of query language can it respond to? In a traditional RDBMS
>>> database one would use SQL; so what is the counterpart query
>>> language with Hbase?
>>
>>
>>
>>
>>
Re: Using SPARQL against HBase
Posted by Tim Robertson <ti...@gmail.com>.
One way would be on each row identified by the Node ID, store the and
Set<ID> holding the IDs of the Nodes the row Node is connected to,
then from any Node you can pull the adjacent Nodes with getByID calls.
Tim
On Mon, Apr 5, 2010 at 9:51 PM, Basmajian, Raffi
<rb...@oppenheimerfunds.com> wrote:
> Using adjacent lists, how did you accomplish the navigation of the list?
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Monday, April 05, 2010 3:16 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Using SPARQL against HBase
>
> Thats right.. We dont delete nodes and want easy navigability, so
> adjacent lists work out well.
>
>
> On Mon, Apr 5, 2010 at 7:53 AM, Tim Robertson
> <ti...@gmail.com>wrote:
>
>> I think he means his table looked like the one on:
>> http://en.wikipedia.org/wiki/Adjacency_list
>>
>> I suspect it means that you can navigate the graph nicely, but a
>> consequence being you might need to update a lot of rows when a node
>> is deleted for example.
>>
>>
>> On Mon, Apr 5, 2010 at 4:42 PM, Basmajian, Raffi
>> <rb...@oppenheimerfunds.com> wrote:
>> > Can you elaborate on what you mean by "adjacent list?" How did you
>> > set that up?
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Amandeep Khurana [mailto:amansk@gmail.com]
>> > Sent: Wednesday, March 31, 2010 5:42 PM
>> > To: hbase-user@hadoop.apache.org
>> > Subject: Re: Using SPARQL against HBase
>> >
>> > I didnt do queries over triples. It was essentially a graph stored
>> > as an adjacency list and used gets and scans for all the work.
>> >
>> > Andrew, if Trend is interested too, we can make this a serious
> project.
>> >
>> >
>> > Amandeep Khurana
>> > Computer Science Graduate Student
>> > University of California, Santa Cruz
>> >
>> >
>> > On Wed, Mar 31, 2010 at 1:08 PM, Basmajian, Raffi <
>> > rbasmajian@oppenheimerfunds.com> wrote:
>> >
>> >> With all of those triples stored in Hbase, how did you query the
> data?
>> >> Using the Hbase Get/Scan api?
>> >>
>> >> -----Original Message-----
>> >> From: Amandeep Khurana [mailto:amansk@gmail.com]
>> >> Sent: Wednesday, March 31, 2010 3:30 PM
>> >> To: hbase-user@hadoop.apache.org; apurtell@apache.org
>> >> Subject: Re: Using SPARQL against HBase
>> >>
>> >> Why do you need to build an in-memory graph which you would want to
>
>> >> read/write to? You could store the graph in HBase directly. As
>> >> pointed
>> >
>> >> out, HBase might not be the best suited for SPARQL queries, but its
>
>> >> not impossible to do. Using the triples, you can form a graph that
>> >> can
>> >
>> >> be represented in HBase as an adjacency list. I've stored graphs
>> >> with 16-17M nodes which was data equivalent to about 600M triples.
>> >> And this
>> >
>> >> was on a small cluster and could certainly scale way more than 16M
>> >> graph nodes.
>> >>
>> >> In case you are interested in working on SPARQL over HBase, we
>> >> could collaborate on it...
>> >>
>> >> -ak
>> >>
>> >>
>> >> Amandeep Khurana
>> >> Computer Science Graduate Student
>> >> University of California, Santa Cruz
>> >>
>> >>
>> >> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
>> >> <ap...@apache.org>wrote:
>> >>
>> >> > Hi Raffi,
>> >> >
>> >> > To read up on fundamentals I suggest Google's BigTable paper:
>> >> > http://labs.google.com/papers/bigtable.html
>> >> >
>> >> > Detail on how HBase implements the BigTable architecture within
>> >> > the Hadoop ecosystem can be found here:
>> >> >
>> >> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
>> >> >
>> >> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.
>> >> > htm
>> >> > l
>> >> >
>> >> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ah
>> >> > ead
>> >> > -l
>> >> > og.html
>> >> >
>> >> > Hope that helps,
>> >> >
>> >> > - Andy
>> >> >
>> >> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
>> >> > > Subject: RE: Using SPARQL against HBase
>> >> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
>> >> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't
>> >> > > respond to SPARQL-like queries, then what type of query
>> >> > > language can it respond
>> >>
>> >> > > to? In a traditional RDBMS database one would use SQL; so what
>> >> > > is the counterpart query language with Hbase?
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >> -------------------------------------------------------------------
>> >> ---
>> >> -------- This e-mail transmission may contain information that is
>> >> proprietary, privileged and/or confidential and is intended
>> >> exclusively for the person(s) to whom it is addressed. Any use,
>> >> copying, retention or disclosure by any person other than the
>> >> intended
>> >
>> >> recipient or the intended recipient's designees is strictly
>> >> prohibited. If you are not the intended recipient or their
>> >> designee, please notify the sender immediately by return e-mail and
>
>> >> delete all copies. OppenheimerFunds may, at its sole discretion,
>> >> monitor, review,
>> >
>> >> retain and/or disclose the content of all email communications.
>> >>
>> >> ===================================================================
>> >> ===
>> >> ========
>> >>
>> >>
>> >
>> >
>> ----------------------------------------------------------------------
>> --------
>> > This e-mail transmission may contain information that is
>> > proprietary,
>> privileged and/or confidential and is intended exclusively for the
>> person(s) to whom it is addressed. Any use, copying, retention or
>> disclosure by any person other than the intended recipient or the
>> intended recipient's designees is strictly prohibited. If you are not
>> the intended recipient or their designee, please notify the sender
>> immediately by return e-mail and delete all copies. OppenheimerFunds
>> may, at its sole discretion, monitor, review, retain and/or disclose
> the content of all email communications.
>> >
>> ======================================================================
>> ========
>> >
>> >
>>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
> ==============================================================================
>
>
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
Using adjacent lists, how did you accomplish the navigation of the list?
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Monday, April 05, 2010 3:16 PM
To: hbase-user@hadoop.apache.org
Subject: Re: Using SPARQL against HBase
Thats right.. We dont delete nodes and want easy navigability, so
adjacent lists work out well.
On Mon, Apr 5, 2010 at 7:53 AM, Tim Robertson
<ti...@gmail.com>wrote:
> I think he means his table looked like the one on:
> http://en.wikipedia.org/wiki/Adjacency_list
>
> I suspect it means that you can navigate the graph nicely, but a
> consequence being you might need to update a lot of rows when a node
> is deleted for example.
>
>
> On Mon, Apr 5, 2010 at 4:42 PM, Basmajian, Raffi
> <rb...@oppenheimerfunds.com> wrote:
> > Can you elaborate on what you mean by "adjacent list?" How did you
> > set that up?
> >
> >
> >
> > -----Original Message-----
> > From: Amandeep Khurana [mailto:amansk@gmail.com]
> > Sent: Wednesday, March 31, 2010 5:42 PM
> > To: hbase-user@hadoop.apache.org
> > Subject: Re: Using SPARQL against HBase
> >
> > I didnt do queries over triples. It was essentially a graph stored
> > as an adjacency list and used gets and scans for all the work.
> >
> > Andrew, if Trend is interested too, we can make this a serious
project.
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
> >
> > On Wed, Mar 31, 2010 at 1:08 PM, Basmajian, Raffi <
> > rbasmajian@oppenheimerfunds.com> wrote:
> >
> >> With all of those triples stored in Hbase, how did you query the
data?
> >> Using the Hbase Get/Scan api?
> >>
> >> -----Original Message-----
> >> From: Amandeep Khurana [mailto:amansk@gmail.com]
> >> Sent: Wednesday, March 31, 2010 3:30 PM
> >> To: hbase-user@hadoop.apache.org; apurtell@apache.org
> >> Subject: Re: Using SPARQL against HBase
> >>
> >> Why do you need to build an in-memory graph which you would want to
> >> read/write to? You could store the graph in HBase directly. As
> >> pointed
> >
> >> out, HBase might not be the best suited for SPARQL queries, but its
> >> not impossible to do. Using the triples, you can form a graph that
> >> can
> >
> >> be represented in HBase as an adjacency list. I've stored graphs
> >> with 16-17M nodes which was data equivalent to about 600M triples.
> >> And this
> >
> >> was on a small cluster and could certainly scale way more than 16M
> >> graph nodes.
> >>
> >> In case you are interested in working on SPARQL over HBase, we
> >> could collaborate on it...
> >>
> >> -ak
> >>
> >>
> >> Amandeep Khurana
> >> Computer Science Graduate Student
> >> University of California, Santa Cruz
> >>
> >>
> >> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> >> <ap...@apache.org>wrote:
> >>
> >> > Hi Raffi,
> >> >
> >> > To read up on fundamentals I suggest Google's BigTable paper:
> >> > http://labs.google.com/papers/bigtable.html
> >> >
> >> > Detail on how HBase implements the BigTable architecture within
> >> > the Hadoop ecosystem can be found here:
> >> >
> >> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> >> >
> >> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.
> >> > htm
> >> > l
> >> >
> >> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ah
> >> > ead
> >> > -l
> >> > og.html
> >> >
> >> > Hope that helps,
> >> >
> >> > - Andy
> >> >
> >> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> >> > > Subject: RE: Using SPARQL against HBase
> >> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> >> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't
> >> > > respond to SPARQL-like queries, then what type of query
> >> > > language can it respond
> >>
> >> > > to? In a traditional RDBMS database one would use SQL; so what
> >> > > is the counterpart query language with Hbase?
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >> -------------------------------------------------------------------
> >> ---
> >> -------- This e-mail transmission may contain information that is
> >> proprietary, privileged and/or confidential and is intended
> >> exclusively for the person(s) to whom it is addressed. Any use,
> >> copying, retention or disclosure by any person other than the
> >> intended
> >
> >> recipient or the intended recipient's designees is strictly
> >> prohibited. If you are not the intended recipient or their
> >> designee, please notify the sender immediately by return e-mail and
> >> delete all copies. OppenheimerFunds may, at its sole discretion,
> >> monitor, review,
> >
> >> retain and/or disclose the content of all email communications.
> >>
> >> ===================================================================
> >> ===
> >> ========
> >>
> >>
> >
> >
> ----------------------------------------------------------------------
> --------
> > This e-mail transmission may contain information that is
> > proprietary,
> privileged and/or confidential and is intended exclusively for the
> person(s) to whom it is addressed. Any use, copying, retention or
> disclosure by any person other than the intended recipient or the
> intended recipient's designees is strictly prohibited. If you are not
> the intended recipient or their designee, please notify the sender
> immediately by return e-mail and delete all copies. OppenheimerFunds
> may, at its sole discretion, monitor, review, retain and/or disclose
the content of all email communications.
> >
> ======================================================================
> ========
> >
> >
>
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
Re: Using SPARQL against HBase
Posted by Amandeep Khurana <am...@gmail.com>.
Thats right.. We dont delete nodes and want easy navigability, so adjacent
lists work out well.
On Mon, Apr 5, 2010 at 7:53 AM, Tim Robertson <ti...@gmail.com>wrote:
> I think he means his table looked like the one on:
> http://en.wikipedia.org/wiki/Adjacency_list
>
> I suspect it means that you can navigate the graph nicely, but a
> consequence being you might need to update a lot of rows when a node
> is deleted for example.
>
>
> On Mon, Apr 5, 2010 at 4:42 PM, Basmajian, Raffi
> <rb...@oppenheimerfunds.com> wrote:
> > Can you elaborate on what you mean by "adjacent list?" How did you set
> > that up?
> >
> >
> >
> > -----Original Message-----
> > From: Amandeep Khurana [mailto:amansk@gmail.com]
> > Sent: Wednesday, March 31, 2010 5:42 PM
> > To: hbase-user@hadoop.apache.org
> > Subject: Re: Using SPARQL against HBase
> >
> > I didnt do queries over triples. It was essentially a graph stored as an
> > adjacency list and used gets and scans for all the work.
> >
> > Andrew, if Trend is interested too, we can make this a serious project.
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
> >
> > On Wed, Mar 31, 2010 at 1:08 PM, Basmajian, Raffi <
> > rbasmajian@oppenheimerfunds.com> wrote:
> >
> >> With all of those triples stored in Hbase, how did you query the data?
> >> Using the Hbase Get/Scan api?
> >>
> >> -----Original Message-----
> >> From: Amandeep Khurana [mailto:amansk@gmail.com]
> >> Sent: Wednesday, March 31, 2010 3:30 PM
> >> To: hbase-user@hadoop.apache.org; apurtell@apache.org
> >> Subject: Re: Using SPARQL against HBase
> >>
> >> Why do you need to build an in-memory graph which you would want to
> >> read/write to? You could store the graph in HBase directly. As pointed
> >
> >> out, HBase might not be the best suited for SPARQL queries, but its
> >> not impossible to do. Using the triples, you can form a graph that can
> >
> >> be represented in HBase as an adjacency list. I've stored graphs with
> >> 16-17M nodes which was data equivalent to about 600M triples. And this
> >
> >> was on a small cluster and could certainly scale way more than 16M
> >> graph nodes.
> >>
> >> In case you are interested in working on SPARQL over HBase, we could
> >> collaborate on it...
> >>
> >> -ak
> >>
> >>
> >> Amandeep Khurana
> >> Computer Science Graduate Student
> >> University of California, Santa Cruz
> >>
> >>
> >> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> >> <ap...@apache.org>wrote:
> >>
> >> > Hi Raffi,
> >> >
> >> > To read up on fundamentals I suggest Google's BigTable paper:
> >> > http://labs.google.com/papers/bigtable.html
> >> >
> >> > Detail on how HBase implements the BigTable architecture within the
> >> > Hadoop ecosystem can be found here:
> >> >
> >> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> >> >
> >> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.htm
> >> > l
> >> >
> >> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead
> >> > -l
> >> > og.html
> >> >
> >> > Hope that helps,
> >> >
> >> > - Andy
> >> >
> >> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> >> > > Subject: RE: Using SPARQL against HBase
> >> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> >> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond
> >> > > to SPARQL-like queries, then what type of query language can it
> >> > > respond
> >>
> >> > > to? In a traditional RDBMS database one would use SQL; so what is
> >> > > the counterpart query language with Hbase?
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >> ----------------------------------------------------------------------
> >> -------- This e-mail transmission may contain information that is
> >> proprietary, privileged and/or confidential and is intended
> >> exclusively for the person(s) to whom it is addressed. Any use,
> >> copying, retention or disclosure by any person other than the intended
> >
> >> recipient or the intended recipient's designees is strictly
> >> prohibited. If you are not the intended recipient or their designee,
> >> please notify the sender immediately by return e-mail and delete all
> >> copies. OppenheimerFunds may, at its sole discretion, monitor, review,
> >
> >> retain and/or disclose the content of all email communications.
> >>
> >> ======================================================================
> >> ========
> >>
> >>
> >
> >
> ------------------------------------------------------------------------------
> > This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the person(s)
> to whom it is addressed. Any use, copying, retention or disclosure by any
> person other than the intended recipient or the intended recipient's
> designees is strictly prohibited. If you are not the intended recipient or
> their designee, please notify the sender immediately by return e-mail and
> delete all copies. OppenheimerFunds may, at its sole discretion, monitor,
> review, retain and/or disclose the content of all email communications.
> >
> ==============================================================================
> >
> >
>
Re: Using SPARQL against HBase
Posted by Tim Robertson <ti...@gmail.com>.
I think he means his table looked like the one on:
http://en.wikipedia.org/wiki/Adjacency_list
I suspect it means that you can navigate the graph nicely, but a
consequence being you might need to update a lot of rows when a node
is deleted for example.
On Mon, Apr 5, 2010 at 4:42 PM, Basmajian, Raffi
<rb...@oppenheimerfunds.com> wrote:
> Can you elaborate on what you mean by "adjacent list?" How did you set
> that up?
>
>
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Wednesday, March 31, 2010 5:42 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Using SPARQL against HBase
>
> I didnt do queries over triples. It was essentially a graph stored as an
> adjacency list and used gets and scans for all the work.
>
> Andrew, if Trend is interested too, we can make this a serious project.
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Wed, Mar 31, 2010 at 1:08 PM, Basmajian, Raffi <
> rbasmajian@oppenheimerfunds.com> wrote:
>
>> With all of those triples stored in Hbase, how did you query the data?
>> Using the Hbase Get/Scan api?
>>
>> -----Original Message-----
>> From: Amandeep Khurana [mailto:amansk@gmail.com]
>> Sent: Wednesday, March 31, 2010 3:30 PM
>> To: hbase-user@hadoop.apache.org; apurtell@apache.org
>> Subject: Re: Using SPARQL against HBase
>>
>> Why do you need to build an in-memory graph which you would want to
>> read/write to? You could store the graph in HBase directly. As pointed
>
>> out, HBase might not be the best suited for SPARQL queries, but its
>> not impossible to do. Using the triples, you can form a graph that can
>
>> be represented in HBase as an adjacency list. I've stored graphs with
>> 16-17M nodes which was data equivalent to about 600M triples. And this
>
>> was on a small cluster and could certainly scale way more than 16M
>> graph nodes.
>>
>> In case you are interested in working on SPARQL over HBase, we could
>> collaborate on it...
>>
>> -ak
>>
>>
>> Amandeep Khurana
>> Computer Science Graduate Student
>> University of California, Santa Cruz
>>
>>
>> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
>> <ap...@apache.org>wrote:
>>
>> > Hi Raffi,
>> >
>> > To read up on fundamentals I suggest Google's BigTable paper:
>> > http://labs.google.com/papers/bigtable.html
>> >
>> > Detail on how HBase implements the BigTable architecture within the
>> > Hadoop ecosystem can be found here:
>> >
>> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
>> >
>> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.htm
>> > l
>> >
>> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead
>> > -l
>> > og.html
>> >
>> > Hope that helps,
>> >
>> > - Andy
>> >
>> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
>> > > Subject: RE: Using SPARQL against HBase
>> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
>> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond
>> > > to SPARQL-like queries, then what type of query language can it
>> > > respond
>>
>> > > to? In a traditional RDBMS database one would use SQL; so what is
>> > > the counterpart query language with Hbase?
>> >
>> >
>> >
>> >
>> >
>>
>>
>> ----------------------------------------------------------------------
>> -------- This e-mail transmission may contain information that is
>> proprietary, privileged and/or confidential and is intended
>> exclusively for the person(s) to whom it is addressed. Any use,
>> copying, retention or disclosure by any person other than the intended
>
>> recipient or the intended recipient's designees is strictly
>> prohibited. If you are not the intended recipient or their designee,
>> please notify the sender immediately by return e-mail and delete all
>> copies. OppenheimerFunds may, at its sole discretion, monitor, review,
>
>> retain and/or disclose the content of all email communications.
>>
>> ======================================================================
>> ========
>>
>>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
> ==============================================================================
>
>
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
Can you elaborate on what you mean by "adjacent list?" How did you set
that up?
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Wednesday, March 31, 2010 5:42 PM
To: hbase-user@hadoop.apache.org
Subject: Re: Using SPARQL against HBase
I didnt do queries over triples. It was essentially a graph stored as an
adjacency list and used gets and scans for all the work.
Andrew, if Trend is interested too, we can make this a serious project.
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 1:08 PM, Basmajian, Raffi <
rbasmajian@oppenheimerfunds.com> wrote:
> With all of those triples stored in Hbase, how did you query the data?
> Using the Hbase Get/Scan api?
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Wednesday, March 31, 2010 3:30 PM
> To: hbase-user@hadoop.apache.org; apurtell@apache.org
> Subject: Re: Using SPARQL against HBase
>
> Why do you need to build an in-memory graph which you would want to
> read/write to? You could store the graph in HBase directly. As pointed
> out, HBase might not be the best suited for SPARQL queries, but its
> not impossible to do. Using the triples, you can form a graph that can
> be represented in HBase as an adjacency list. I've stored graphs with
> 16-17M nodes which was data equivalent to about 600M triples. And this
> was on a small cluster and could certainly scale way more than 16M
> graph nodes.
>
> In case you are interested in working on SPARQL over HBase, we could
> collaborate on it...
>
> -ak
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> <ap...@apache.org>wrote:
>
> > Hi Raffi,
> >
> > To read up on fundamentals I suggest Google's BigTable paper:
> > http://labs.google.com/papers/bigtable.html
> >
> > Detail on how HBase implements the BigTable architecture within the
> > Hadoop ecosystem can be found here:
> >
> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> >
> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.htm
> > l
> >
> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead
> > -l
> > og.html
> >
> > Hope that helps,
> >
> > - Andy
> >
> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > > Subject: RE: Using SPARQL against HBase
> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond
> > > to SPARQL-like queries, then what type of query language can it
> > > respond
>
> > > to? In a traditional RDBMS database one would use SQL; so what is
> > > the counterpart query language with Hbase?
> >
> >
> >
> >
> >
>
>
> ----------------------------------------------------------------------
> -------- This e-mail transmission may contain information that is
> proprietary, privileged and/or confidential and is intended
> exclusively for the person(s) to whom it is addressed. Any use,
> copying, retention or disclosure by any person other than the intended
> recipient or the intended recipient's designees is strictly
> prohibited. If you are not the intended recipient or their designee,
> please notify the sender immediately by return e-mail and delete all
> copies. OppenheimerFunds may, at its sole discretion, monitor, review,
> retain and/or disclose the content of all email communications.
>
> ======================================================================
> ========
>
>
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
Re: Using SPARQL against HBase
Posted by Amandeep Khurana <am...@gmail.com>.
I didnt do queries over triples. It was essentially a graph stored as an
adjacency list and used gets and scans for all the work.
Andrew, if Trend is interested too, we can make this a serious project.
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 1:08 PM, Basmajian, Raffi <
rbasmajian@oppenheimerfunds.com> wrote:
> With all of those triples stored in Hbase, how did you query the data?
> Using the Hbase Get/Scan api?
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Wednesday, March 31, 2010 3:30 PM
> To: hbase-user@hadoop.apache.org; apurtell@apache.org
> Subject: Re: Using SPARQL against HBase
>
> Why do you need to build an in-memory graph which you would want to
> read/write to? You could store the graph in HBase directly. As pointed
> out, HBase might not be the best suited for SPARQL queries, but its not
> impossible to do. Using the triples, you can form a graph that can be
> represented in HBase as an adjacency list. I've stored graphs with
> 16-17M nodes which was data equivalent to about 600M triples. And this
> was on a small cluster and could certainly scale way more than 16M graph
> nodes.
>
> In case you are interested in working on SPARQL over HBase, we could
> collaborate on it...
>
> -ak
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> <ap...@apache.org>wrote:
>
> > Hi Raffi,
> >
> > To read up on fundamentals I suggest Google's BigTable paper:
> > http://labs.google.com/papers/bigtable.html
> >
> > Detail on how HBase implements the BigTable architecture within the
> > Hadoop ecosystem can be found here:
> >
> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >
> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> > og.html
> >
> > Hope that helps,
> >
> > - Andy
> >
> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > > Subject: RE: Using SPARQL against HBase
> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > > SPARQL-like queries, then what type of query language can it respond
>
> > > to? In a traditional RDBMS database one would use SQL; so what is
> > > the counterpart query language with Hbase?
> >
> >
> >
> >
> >
>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the person(s)
> to whom it is addressed. Any use, copying, retention or disclosure by any
> person other than the intended recipient or the intended recipient's
> designees is strictly prohibited. If you are not the intended recipient or
> their designee, please notify the sender immediately by return e-mail and
> delete all copies. OppenheimerFunds may, at its sole discretion, monitor,
> review, retain and/or disclose the content of all email communications.
>
> ==============================================================================
>
>
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
With all of those triples stored in Hbase, how did you query the data?
Using the Hbase Get/Scan api?
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Wednesday, March 31, 2010 3:30 PM
To: hbase-user@hadoop.apache.org; apurtell@apache.org
Subject: Re: Using SPARQL against HBase
Why do you need to build an in-memory graph which you would want to
read/write to? You could store the graph in HBase directly. As pointed
out, HBase might not be the best suited for SPARQL queries, but its not
impossible to do. Using the triples, you can form a graph that can be
represented in HBase as an adjacency list. I've stored graphs with
16-17M nodes which was data equivalent to about 600M triples. And this
was on a small cluster and could certainly scale way more than 16M graph
nodes.
In case you are interested in working on SPARQL over HBase, we could
collaborate on it...
-ak
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
<ap...@apache.org>wrote:
> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper:
> http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the
> Hadoop ecosystem can be found here:
>
> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> og.html
>
> Hope that helps,
>
> - Andy
>
> > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > SPARQL-like queries, then what type of query language can it respond
> > to? In a traditional RDBMS database one would use SQL; so what is
> > the counterpart query language with Hbase?
>
>
>
>
>
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
Re: Using SPARQL against HBase
Posted by vi...@nokia.com.
I am very interested in participating.
Victor
On 4/1/10 5:45 PM, "ext Amandeep Khurana" <am...@gmail.com> wrote:
Andrew and I just had a chat about exploring how we can leverage HBase for a
scalable RDF store and we'll be looking at it in more detail over the next
few days. Is anyone of you interested in helping out? We are going to be
looking at what all is required to build a triple store + query engine on
HBase and how HBase can be used as is or remodeled to fit the problem.
Depending on what we find out, we'll decide on taking the project further
and committing efforts towards it.
-Amandeep
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Apr 1, 2010 at 1:12 PM, Jürgen Jakobitsch <ja...@punkt.at>wrote:
> hi,
>
> this sounds very interesting to me, i'm currently fiddling
> around with a suitable row and column setup for triples.
>
> i'm about to implement openrdf's sail api for hbase (i just did
> a lucene quad store implementation which is superfast a scales
> to a couple of hundreds of millions of triples (http://turnguard.com/tuqs
> ))
> but i'm in my first days of hbase encounters, so my experience
> in row column design is manageable.
>
> from my point of view the problem is to really efficiantly store
> besides the triples themselves the contexts (named graphs) and
> languages of literal.
>
> by the way : i just did a small tablemanager (in beta) that lets
> you create htables -> from <- rdf (see
> http://sourceforge.net/projects/hbasetablemgr/)
>
> i'd be really happy to contribute on the rdf and sparql side,
> but certainly could need some help on the hbase table design side.
>
> wkr www.turnguard.com/turnguard
>
>
>
> ----- Original Message -----
> From: "Raffi Basmajian" <rb...@oppenheimerfunds.com>
> To: hbase-user@hadoop.apache.org, apurtell@apache.org
> Sent: Thursday, April 1, 2010 9:45:59 PM
> Subject: RE: Using SPARQL against HBase
>
>
> This is an interesting article from a few guys over at BBN/Raytheon. By
> storing triples in flat files theu used a custom algorithm, detailed in
> the article, to iterate the WHERE clause from a SPARQL query and reduce
> the map into the desired result.
>
> This is very similar to what I need to do; the only difference being
> that our data is stored in Hbase tables, not as triples in flat files.
>
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Wednesday, March 31, 2010 3:30 PM
> To: hbase-user@hadoop.apache.org; apurtell@apache.org
> Subject: Re: Using SPARQL against HBase
>
> Why do you need to build an in-memory graph which you would want to
> read/write to? You could store the graph in HBase directly. As pointed
> out, HBase might not be the best suited for SPARQL queries, but its not
> impossible to do. Using the triples, you can form a graph that can be
> represented in HBase as an adjacency list. I've stored graphs with
> 16-17M nodes which was data equivalent to about 600M triples. And this
> was on a small cluster and could certainly scale way more than 16M graph
> nodes.
>
> In case you are interested in working on SPARQL over HBase, we could
> collaborate on it...
>
> -ak
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> <ap...@apache.org>wrote:
>
> > Hi Raffi,
> >
> > To read up on fundamentals I suggest Google's BigTable paper:
> > http://labs.google.com/papers/bigtable.html
> >
> > Detail on how HBase implements the BigTable architecture within the
> > Hadoop ecosystem can be found here:
> >
> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >
> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> > og.html
> >
> > Hope that helps,
> >
> > - Andy
> >
> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > > Subject: RE: Using SPARQL against HBase
> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > > SPARQL-like queries, then what type of query language can it respond
>
> > > to? In a traditional RDBMS database one would use SQL; so what is
> > > the counterpart query language with Hbase?
> >
> >
> >
> >
> >
>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the person(s)
> to whom it is addressed. Any use, copying, retention or disclosure by any
> person other than the intended recipient or the intended recipient's
> designees is strictly prohibited. If you are not the intended recipient or
> their designee, please notify the sender immediately by return e-mail and
> delete all copies. OppenheimerFunds may, at its sole discretion, monitor,
> review, retain and/or disclose the content of all email communications.
>
> ==============================================================================
>
>
> --
> punkt. netServices
> ______________________________
> Jürgen Jakobitsch
> Codeography
>
> Lerchenfelder Gürtel 43 Top 5/2
> A - 1160 Wien
> Tel.: 01 / 897 41 22 - 29
> Fax: 01 / 897 41 22 - 22
>
> netServices http://www.punkt.at
>
>
Re: Using SPARQL against HBase
Posted by Amandeep Khurana <am...@gmail.com>.
Andrew and I just had a chat about exploring how we can leverage HBase for a
scalable RDF store and we'll be looking at it in more detail over the next
few days. Is anyone of you interested in helping out? We are going to be
looking at what all is required to build a triple store + query engine on
HBase and how HBase can be used as is or remodeled to fit the problem.
Depending on what we find out, we'll decide on taking the project further
and committing efforts towards it.
-Amandeep
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Thu, Apr 1, 2010 at 1:12 PM, Jürgen Jakobitsch <ja...@punkt.at>wrote:
> hi,
>
> this sounds very interesting to me, i'm currently fiddling
> around with a suitable row and column setup for triples.
>
> i'm about to implement openrdf's sail api for hbase (i just did
> a lucene quad store implementation which is superfast a scales
> to a couple of hundreds of millions of triples (http://turnguard.com/tuqs
> ))
> but i'm in my first days of hbase encounters, so my experience
> in row column design is manageable.
>
> from my point of view the problem is to really efficiantly store
> besides the triples themselves the contexts (named graphs) and
> languages of literal.
>
> by the way : i just did a small tablemanager (in beta) that lets
> you create htables -> from <- rdf (see
> http://sourceforge.net/projects/hbasetablemgr/)
>
> i'd be really happy to contribute on the rdf and sparql side,
> but certainly could need some help on the hbase table design side.
>
> wkr www.turnguard.com/turnguard
>
>
>
> ----- Original Message -----
> From: "Raffi Basmajian" <rb...@oppenheimerfunds.com>
> To: hbase-user@hadoop.apache.org, apurtell@apache.org
> Sent: Thursday, April 1, 2010 9:45:59 PM
> Subject: RE: Using SPARQL against HBase
>
>
> This is an interesting article from a few guys over at BBN/Raytheon. By
> storing triples in flat files theu used a custom algorithm, detailed in
> the article, to iterate the WHERE clause from a SPARQL query and reduce
> the map into the desired result.
>
> This is very similar to what I need to do; the only difference being
> that our data is stored in Hbase tables, not as triples in flat files.
>
>
> -----Original Message-----
> From: Amandeep Khurana [mailto:amansk@gmail.com]
> Sent: Wednesday, March 31, 2010 3:30 PM
> To: hbase-user@hadoop.apache.org; apurtell@apache.org
> Subject: Re: Using SPARQL against HBase
>
> Why do you need to build an in-memory graph which you would want to
> read/write to? You could store the graph in HBase directly. As pointed
> out, HBase might not be the best suited for SPARQL queries, but its not
> impossible to do. Using the triples, you can form a graph that can be
> represented in HBase as an adjacency list. I've stored graphs with
> 16-17M nodes which was data equivalent to about 600M triples. And this
> was on a small cluster and could certainly scale way more than 16M graph
> nodes.
>
> In case you are interested in working on SPARQL over HBase, we could
> collaborate on it...
>
> -ak
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
> <ap...@apache.org>wrote:
>
> > Hi Raffi,
> >
> > To read up on fundamentals I suggest Google's BigTable paper:
> > http://labs.google.com/papers/bigtable.html
> >
> > Detail on how HBase implements the BigTable architecture within the
> > Hadoop ecosystem can be found here:
> >
> > http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> > http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
> >
> > http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> > og.html
> >
> > Hope that helps,
> >
> > - Andy
> >
> > > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > > Subject: RE: Using SPARQL against HBase
> > > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > > SPARQL-like queries, then what type of query language can it respond
>
> > > to? In a traditional RDBMS database one would use SQL; so what is
> > > the counterpart query language with Hbase?
> >
> >
> >
> >
> >
>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the person(s)
> to whom it is addressed. Any use, copying, retention or disclosure by any
> person other than the intended recipient or the intended recipient's
> designees is strictly prohibited. If you are not the intended recipient or
> their designee, please notify the sender immediately by return e-mail and
> delete all copies. OppenheimerFunds may, at its sole discretion, monitor,
> review, retain and/or disclose the content of all email communications.
>
> ==============================================================================
>
>
> --
> punkt. netServices
> ______________________________
> Jürgen Jakobitsch
> Codeography
>
> Lerchenfelder Gürtel 43 Top 5/2
> A - 1160 Wien
> Tel.: 01 / 897 41 22 - 29
> Fax: 01 / 897 41 22 - 22
>
> netServices http://www.punkt.at
>
>
Re: Using SPARQL against HBase
Posted by Jürgen Jakobitsch <ja...@punkt.at>.
hi,
this sounds very interesting to me, i'm currently fiddling
around with a suitable row and column setup for triples.
i'm about to implement openrdf's sail api for hbase (i just did
a lucene quad store implementation which is superfast a scales
to a couple of hundreds of millions of triples (http://turnguard.com/tuqs))
but i'm in my first days of hbase encounters, so my experience
in row column design is manageable.
from my point of view the problem is to really efficiantly store
besides the triples themselves the contexts (named graphs) and
languages of literal.
by the way : i just did a small tablemanager (in beta) that lets
you create htables -> from <- rdf (see http://sourceforge.net/projects/hbasetablemgr/)
i'd be really happy to contribute on the rdf and sparql side,
but certainly could need some help on the hbase table design side.
wkr www.turnguard.com/turnguard
----- Original Message -----
From: "Raffi Basmajian" <rb...@oppenheimerfunds.com>
To: hbase-user@hadoop.apache.org, apurtell@apache.org
Sent: Thursday, April 1, 2010 9:45:59 PM
Subject: RE: Using SPARQL against HBase
This is an interesting article from a few guys over at BBN/Raytheon. By
storing triples in flat files theu used a custom algorithm, detailed in
the article, to iterate the WHERE clause from a SPARQL query and reduce
the map into the desired result.
This is very similar to what I need to do; the only difference being
that our data is stored in Hbase tables, not as triples in flat files.
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Wednesday, March 31, 2010 3:30 PM
To: hbase-user@hadoop.apache.org; apurtell@apache.org
Subject: Re: Using SPARQL against HBase
Why do you need to build an in-memory graph which you would want to
read/write to? You could store the graph in HBase directly. As pointed
out, HBase might not be the best suited for SPARQL queries, but its not
impossible to do. Using the triples, you can form a graph that can be
represented in HBase as an adjacency list. I've stored graphs with
16-17M nodes which was data equivalent to about 600M triples. And this
was on a small cluster and could certainly scale way more than 16M graph
nodes.
In case you are interested in working on SPARQL over HBase, we could
collaborate on it...
-ak
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
<ap...@apache.org>wrote:
> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper:
> http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the
> Hadoop ecosystem can be found here:
>
> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> og.html
>
> Hope that helps,
>
> - Andy
>
> > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > SPARQL-like queries, then what type of query language can it respond
> > to? In a traditional RDBMS database one would use SQL; so what is
> > the counterpart query language with Hbase?
>
>
>
>
>
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
--
punkt. netServices
______________________________
Jürgen Jakobitsch
Codeography
Lerchenfelder Gürtel 43 Top 5/2
A - 1160 Wien
Tel.: 01 / 897 41 22 - 29
Fax: 01 / 897 41 22 - 22
netServices http://www.punkt.at
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
This is an interesting article from a few guys over at BBN/Raytheon. By
storing triples in flat files theu used a custom algorithm, detailed in
the article, to iterate the WHERE clause from a SPARQL query and reduce
the map into the desired result.
This is very similar to what I need to do; the only difference being
that our data is stored in Hbase tables, not as triples in flat files.
-----Original Message-----
From: Amandeep Khurana [mailto:amansk@gmail.com]
Sent: Wednesday, March 31, 2010 3:30 PM
To: hbase-user@hadoop.apache.org; apurtell@apache.org
Subject: Re: Using SPARQL against HBase
Why do you need to build an in-memory graph which you would want to
read/write to? You could store the graph in HBase directly. As pointed
out, HBase might not be the best suited for SPARQL queries, but its not
impossible to do. Using the triples, you can form a graph that can be
represented in HBase as an adjacency list. I've stored graphs with
16-17M nodes which was data equivalent to about 600M triples. And this
was on a small cluster and could certainly scale way more than 16M graph
nodes.
In case you are interested in working on SPARQL over HBase, we could
collaborate on it...
-ak
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell
<ap...@apache.org>wrote:
> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper:
> http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the
> Hadoop ecosystem can be found here:
>
> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-l
> og.html
>
> Hope that helps,
>
> - Andy
>
> > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM If Hbase can't respond to
> > SPARQL-like queries, then what type of query language can it respond
> > to? In a traditional RDBMS database one would use SQL; so what is
> > the counterpart query language with Hbase?
>
>
>
>
>
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
Re: Using SPARQL against HBase
Posted by Amandeep Khurana <am...@gmail.com>.
Why do you need to build an in-memory graph which you would want to
read/write to? You could store the graph in HBase directly. As pointed out,
HBase might not be the best suited for SPARQL queries, but its not
impossible to do. Using the triples, you can form a graph that can be
represented in HBase as an adjacency list. I've stored graphs with 16-17M
nodes which was data equivalent to about 600M triples. And this was on a
small cluster and could certainly scale way more than 16M graph nodes.
In case you are interested in working on SPARQL over HBase, we could
collaborate on it...
-ak
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 11:56 AM, Andrew Purtell <ap...@apache.org>wrote:
> Hi Raffi,
>
> To read up on fundamentals I suggest Google's BigTable paper:
> http://labs.google.com/papers/bigtable.html
>
> Detail on how HBase implements the BigTable architecture within the Hadoop
> ecosystem can be found here:
>
> http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
> http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
>
> Hope that helps,
>
> - Andy
>
> > From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> > Subject: RE: Using SPARQL against HBase
> > To: hbase-user@hadoop.apache.org, apurtell@apache.org
> > Date: Wednesday, March 31, 2010, 11:42 AM
> > If Hbase can't respond to SPARQL-like queries, then what type
> > of query language can it respond to? In a traditional RDBMS
> > database one would use SQL; so what is the counterpart query
> > language with Hbase?
>
>
>
>
>
RE: Using SPARQL against HBase
Posted by Andrew Purtell <ap...@apache.org>.
Hi Raffi,
To read up on fundamentals I suggest Google's BigTable paper: http://labs.google.com/papers/bigtable.html
Detail on how HBase implements the BigTable architecture within the Hadoop ecosystem can be found here:
http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture
http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
Hope that helps,
- Andy
> From: Basmajian, Raffi <rb...@oppenheimerfunds.com>
> Subject: RE: Using SPARQL against HBase
> To: hbase-user@hadoop.apache.org, apurtell@apache.org
> Date: Wednesday, March 31, 2010, 11:42 AM
> If Hbase can't respond to SPARQL-like queries, then what type
> of query language can it respond to? In a traditional RDBMS
> database one would use SQL; so what is the counterpart query
> language with Hbase?
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
If Hbase can't respond to SPARQL-like queries, then what type of query
language can it respond to? In a traditional RDBMS database one would
use SQL; so what is the counterpart query language with Hbase?
-RNY
-----Original Message-----
From: Andrew Purtell [mailto:apurtell@apache.org]
Sent: Wednesday, March 31, 2010 12:27 PM
To: hbase-user@hadoop.apache.org
Subject: Re: Using SPARQL against HBase
HBase has nice properties for efficiently storing, for example, a sparse
adjacency representation of a graph, very large graphs. I'm sure it
could be used to store an enormous number of RDF triples. But this is a
long long way from something that can respond to SPARQL queries. The RDF
store I've seen that most closely resembles HBase is "bigdata":
http://www.bigdata.com/bigdata/blog/. I've only heard of it, not looked
at it.
> From: Stack
>
> The in-memory graph would hold all data or just metadata?
> You might look at IHBase in the 'indexed' contrib to see how it loads
> an index on region open (It subclasses hbase so it can catch key
> transistions)
>
> Why HBase and not a native graph database?
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
RE: Using SPARQL against HBase
Posted by "Basmajian, Raffi" <rb...@oppenheimerfunds.com>.
It is my understanding that HEART is abandoned at the moment; no
activity. I checked the site and, while there is some source code
available, it would be difficult to determine the state of the features
in the code without knowing how far the project has progressed with
respect to the original goal.
-----Original Message-----
From: Andrew Purtell [mailto:apurtell@apache.org]
Sent: Wednesday, March 31, 2010 2:28 PM
To: hbase-user@hadoop.apache.org; Jonathan Gray
Subject: RE: Using SPARQL against HBase
I thought Heart was dead.
- Andy
> From: Jonathan Gray
> Subject: RE: Using SPARQL against HBase Stack pointed this out to me
> yesterday which could be of interest to you:
>
> http://wiki.apache.org/incubator/HeartProposal
>
> http://heart.korea.ac.kr/
------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================
RE: Using SPARQL against HBase
Posted by Andrew Purtell <ap...@apache.org>.
I thought Heart was dead.
- Andy
> From: Jonathan Gray
> Subject: RE: Using SPARQL against HBase
> Stack pointed this out to me yesterday which could be of interest to you:
>
> http://wiki.apache.org/incubator/HeartProposal
>
> http://heart.korea.ac.kr/
RE: Using SPARQL against HBase
Posted by Jonathan Gray <jg...@facebook.com>.
Stack pointed this out to me yesterday which could be of interest to you:
http://wiki.apache.org/incubator/HeartProposal
http://heart.korea.ac.kr/
> -----Original Message-----
> From: Andrew Purtell [mailto:apurtell@apache.org]
> Sent: Wednesday, March 31, 2010 9:27 AM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Using SPARQL against HBase
>
> HBase has nice properties for efficiently storing, for example, a
> sparse adjacency representation of a graph, very large graphs. I'm sure
> it could be used to store an enormous number of RDF triples. But this
> is a long long way from something that can respond to SPARQL queries.
> The RDF store I've seen that most closely resembles HBase is "bigdata":
> http://www.bigdata.com/bigdata/blog/. I've only heard of it, not looked
> at it.
>
> > From: Stack
> >
> > The in-memory graph would hold all data or just metadata?
> > You might look at IHBase in the 'indexed' contrib to see how
> > it loads an index on region open (It subclasses hbase so it
> > can catch key transistions)
> >
> > Why HBase and not a native graph database?
>
>
>
>
Re: Using SPARQL against HBase
Posted by Andrew Purtell <ap...@apache.org>.
HBase has nice properties for efficiently storing, for example, a sparse adjacency representation of a graph, very large graphs. I'm sure it could be used to store an enormous number of RDF triples. But this is a long long way from something that can respond to SPARQL queries. The RDF store I've seen that most closely resembles HBase is "bigdata": http://www.bigdata.com/bigdata/blog/. I've only heard of it, not looked at it.
> From: Stack
>
> The in-memory graph would hold all data or just metadata?
> You might look at IHBase in the 'indexed' contrib to see how
> it loads an index on region open (It subclasses hbase so it
> can catch key transistions)
>
> Why HBase and not a native graph database?
Re: Using SPARQL against HBase
Posted by Stack <st...@duboce.net>.
Writes would update your in-memory graph and the backing hbase store?
The in-memory graph would hold all data or just metadata? You might
look at IHBase in the 'indexed' contrib to see how it loads an index
on region open (It subclasses hbase so it can catch key transistions)
Why HBase and not a native graph database?
Yours,
St.Ack
On Wed, Mar 31, 2010 at 8:27 AM, Basmajian, Raffi
<rb...@oppenheimerfunds.com> wrote:
> We are currently researching how to use SPARQL against data in Hbase. I
> understand the use of Get and Scan classes in the Hbase API, but these
> search classes do not return data in the same way SPARQL against RDF
> data returns it. My colleagues and I were discussing that these types of
> search results will require creating an in-memory graph first from
> Hbase, then using SPARQL against that graph. We are not sure how this is
> accomplished. Any advice would help, thank you
>
> -RNY
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
> ==============================================================================
>
Re: Using SPARQL against HBase
Posted by Amandeep Khurana <am...@gmail.com>.
Raffi,
This article might interest you:
http://decentralyze.com/2010/03/09/rdf-meets-nosql/
Amandeep
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Wed, Mar 31, 2010 at 8:27 AM, Basmajian, Raffi <
rbasmajian@oppenheimerfunds.com> wrote:
> We are currently researching how to use SPARQL against data in Hbase. I
> understand the use of Get and Scan classes in the Hbase API, but these
> search classes do not return data in the same way SPARQL against RDF
> data returns it. My colleagues and I were discussing that these types of
> search results will require creating an in-memory graph first from
> Hbase, then using SPARQL against that graph. We are not sure how this is
> accomplished. Any advice would help, thank you
>
> -RNY
>
>
> ------------------------------------------------------------------------------
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the person(s)
> to whom it is addressed. Any use, copying, retention or disclosure by any
> person other than the intended recipient or the intended recipient's
> designees is strictly prohibited. If you are not the intended recipient or
> their designee, please notify the sender immediately by return e-mail and
> delete all copies. OppenheimerFunds may, at its sole discretion, monitor,
> review, retain and/or disclose the content of all email communications.
>
> ==============================================================================
>