You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Flavio Pompermaier <po...@okkam.it> on 2014/06/19 16:49:44 UTC
Spark and RDF
Hi guys,
I'm analyzing the possibility to use Spark to analyze RDF files and define
reusable Shark operators on them (custom filtering, transforming,
aggregating, etc). Is that possible? Any hint?
Best,
Flavio
Re: Spark and RDF
Posted by andy petrella <an...@gmail.com>.
yep, would be cool. Even though sparql has its drawbacks (vs cypher vs
gremlin I mean), however still cool for semantic thingies and c°.
aℕdy ℙetrella
about.me/noootsab
[image: aℕdy ℙetrella on about.me]
<http://about.me/noootsab>
On Fri, Jun 20, 2014 at 10:03 PM, Mayur Rustagi <ma...@gmail.com>
wrote:
> or a seperate RDD for sparql operations ala SchemaRDD .. operators for
> sparql can be defined thr.. not a bad idea :)
>
> Mayur Rustagi
> Ph: +1 (760) 203 3257
> http://www.sigmoidanalytics.com
> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>
>
>
> On Fri, Jun 20, 2014 at 3:56 PM, andy petrella <an...@gmail.com>
> wrote:
>
>> Maybe some SPARQL features in Shark, then ?
>>
>> aℕdy ℙetrella
>> about.me/noootsab
>> [image: aℕdy ℙetrella on about.me]
>>
>> <http://about.me/noootsab>
>>
>>
>> On Fri, Jun 20, 2014 at 9:45 PM, Mayur Rustagi <ma...@gmail.com>
>> wrote:
>>
>>> You are looking to create Shark operators for RDF? Since Shark backend
>>> is shifting to SparkSQL it would be slightly hard but much better effort
>>> would be to shift Gremlin to Spark (though a much beefier one :) )
>>>
>>> Mayur Rustagi
>>> Ph: +1 (760) 203 3257
>>> http://www.sigmoidanalytics.com
>>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>>
>>>
>>>
>>> On Fri, Jun 20, 2014 at 3:39 PM, andy petrella <an...@gmail.com>
>>> wrote:
>>>
>>>> For RDF, may GraphX be particularly approriated?
>>>>
>>>> aℕdy ℙetrella
>>>> about.me/noootsab
>>>> [image: aℕdy ℙetrella on about.me]
>>>>
>>>> <http://about.me/noootsab>
>>>>
>>>>
>>>> On Thu, Jun 19, 2014 at 4:49 PM, Flavio Pompermaier <
>>>> pompermaier@okkam.it> wrote:
>>>>
>>>>> Hi guys,
>>>>>
>>>>> I'm analyzing the possibility to use Spark to analyze RDF files and
>>>>> define reusable Shark operators on them (custom filtering, transforming,
>>>>> aggregating, etc). Is that possible? Any hint?
>>>>>
>>>>> Best,
>>>>> Flavio
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Spark and RDF
Posted by Mayur Rustagi <ma...@gmail.com>.
or a seperate RDD for sparql operations ala SchemaRDD .. operators for
sparql can be defined thr.. not a bad idea :)
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>
On Fri, Jun 20, 2014 at 3:56 PM, andy petrella <an...@gmail.com>
wrote:
> Maybe some SPARQL features in Shark, then ?
>
> aℕdy ℙetrella
> about.me/noootsab
> [image: aℕdy ℙetrella on about.me]
>
> <http://about.me/noootsab>
>
>
> On Fri, Jun 20, 2014 at 9:45 PM, Mayur Rustagi <ma...@gmail.com>
> wrote:
>
>> You are looking to create Shark operators for RDF? Since Shark backend is
>> shifting to SparkSQL it would be slightly hard but much better effort would
>> be to shift Gremlin to Spark (though a much beefier one :) )
>>
>> Mayur Rustagi
>> Ph: +1 (760) 203 3257
>> http://www.sigmoidanalytics.com
>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>
>>
>>
>> On Fri, Jun 20, 2014 at 3:39 PM, andy petrella <an...@gmail.com>
>> wrote:
>>
>>> For RDF, may GraphX be particularly approriated?
>>>
>>> aℕdy ℙetrella
>>> about.me/noootsab
>>> [image: aℕdy ℙetrella on about.me]
>>>
>>> <http://about.me/noootsab>
>>>
>>>
>>> On Thu, Jun 19, 2014 at 4:49 PM, Flavio Pompermaier <
>>> pompermaier@okkam.it> wrote:
>>>
>>>> Hi guys,
>>>>
>>>> I'm analyzing the possibility to use Spark to analyze RDF files and
>>>> define reusable Shark operators on them (custom filtering, transforming,
>>>> aggregating, etc). Is that possible? Any hint?
>>>>
>>>> Best,
>>>> Flavio
>>>>
>>>
>>>
>>
>
Re: Spark and RDF
Posted by andy petrella <an...@gmail.com>.
Maybe some SPARQL features in Shark, then ?
aℕdy ℙetrella
about.me/noootsab
[image: aℕdy ℙetrella on about.me]
<http://about.me/noootsab>
On Fri, Jun 20, 2014 at 9:45 PM, Mayur Rustagi <ma...@gmail.com>
wrote:
> You are looking to create Shark operators for RDF? Since Shark backend is
> shifting to SparkSQL it would be slightly hard but much better effort would
> be to shift Gremlin to Spark (though a much beefier one :) )
>
> Mayur Rustagi
> Ph: +1 (760) 203 3257
> http://www.sigmoidanalytics.com
> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>
>
>
> On Fri, Jun 20, 2014 at 3:39 PM, andy petrella <an...@gmail.com>
> wrote:
>
>> For RDF, may GraphX be particularly approriated?
>>
>> aℕdy ℙetrella
>> about.me/noootsab
>> [image: aℕdy ℙetrella on about.me]
>>
>> <http://about.me/noootsab>
>>
>>
>> On Thu, Jun 19, 2014 at 4:49 PM, Flavio Pompermaier <pompermaier@okkam.it
>> > wrote:
>>
>>> Hi guys,
>>>
>>> I'm analyzing the possibility to use Spark to analyze RDF files and
>>> define reusable Shark operators on them (custom filtering, transforming,
>>> aggregating, etc). Is that possible? Any hint?
>>>
>>> Best,
>>> Flavio
>>>
>>
>>
>
Re: Spark and RDF
Posted by Mayur Rustagi <ma...@gmail.com>.
You are looking to create Shark operators for RDF? Since Shark backend is
shifting to SparkSQL it would be slightly hard but much better effort would
be to shift Gremlin to Spark (though a much beefier one :) )
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>
On Fri, Jun 20, 2014 at 3:39 PM, andy petrella <an...@gmail.com>
wrote:
> For RDF, may GraphX be particularly approriated?
>
> aℕdy ℙetrella
> about.me/noootsab
> [image: aℕdy ℙetrella on about.me]
>
> <http://about.me/noootsab>
>
>
> On Thu, Jun 19, 2014 at 4:49 PM, Flavio Pompermaier <po...@okkam.it>
> wrote:
>
>> Hi guys,
>>
>> I'm analyzing the possibility to use Spark to analyze RDF files and
>> define reusable Shark operators on them (custom filtering, transforming,
>> aggregating, etc). Is that possible? Any hint?
>>
>> Best,
>> Flavio
>>
>
>
Re: Spark and RDF
Posted by andy petrella <an...@gmail.com>.
For RDF, may GraphX be particularly approriated?
aℕdy ℙetrella
about.me/noootsab
[image: aℕdy ℙetrella on about.me]
<http://about.me/noootsab>
On Thu, Jun 19, 2014 at 4:49 PM, Flavio Pompermaier <po...@okkam.it>
wrote:
> Hi guys,
>
> I'm analyzing the possibility to use Spark to analyze RDF files and define
> reusable Shark operators on them (custom filtering, transforming,
> aggregating, etc). Is that possible? Any hint?
>
> Best,
> Flavio
>