You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Maja-Olivia Lenz <le...@fhi-berlin.mpg.de> on 2020/08/06 14:20:07 UTC
Custom SPARQL functions in Javascript and python
Hi all,
I have read that it's possible in Jena to write custom functions for
SPARQL in Java or Javascript. Is it possible to interface this with
existing python functions? My idea is to start a python server (e.g.
flask) and send post requests through Javascript to obtain the
quantities I need. Does this make sense/ would that work? Are there
better ways? Has anyone experience with Javascript/Python integration?
If I start a SPARQL Query that calls this function very often (say
1000/minute) how worse does performance become?
I don't know whether it matters: So far I've only used ARQ on rdf files
or a local tdb database, however, in the future I'd also like to use Fuseki.
Thanks in advance,
Maja
--
----------------------------------------------------------------------
Maja-Olivia Lenz
Fritz-Haber-Institut der Max-Planck-Gesellschaft
Faradayweg 4-6
D-14195 Berlin
E-Mail: lenz@fhi-berlin.mpg.de
---------------------key:1-0.0735-11600-23.05:fhi--------------------
Re: Custom SPARQL functions in Javascript and python
Posted by Pavel Mikhailovskii <pa...@gmail.com>.
I would propose the following:
Migrate from Nashorn to GraalVM-powered JavaScript engine and include all the necessary dependencies into jena-arq. I've implemented that already; all tests are passing
Implement support for custom scripting extensions using Java Scripting API (JSR-223) without bundling any specific engine. Calling http://jena.apache.org/ARQ/scripting#jython.foo would instantiate Jython scripting engine (if its present on the class path) and invoke function foo, etc
Pavel Mikhailovskii
pavel.mikhailovskii@gmail.com (mailto:pavel.mikhailovskii@gmail.com)
https://thehyve.nl
On Aug 26 2020, at 4:53 pm, Andy Seaborne <an...@apache.org> wrote:
> GraalVM seems the natural evolution.
>
> GraalVM could also be used to compile a fast-startup version of the
> command line tools. It can act as an ahead-of-time (i.e. traditional)
> compiler.
>
> On 17/08/2020 12:00, Pavel Mikhailovskii wrote:
> > Hi Andy,
> >
> > By the way, how are you going to tackle the upcoming removal of Nashorn (https://openjdk.java.net/jeps/372)? One option would be to use GraalVM Polyglot API.
> > As a quick fix, JSEngine.build could just fall back to Rhino.
> > If we agree on the approach, I can start working on that.
>
> > We could even think of supporting other languages than JavaScript, for example Python and R.
> That would be interesting
> The JS for SPARQL takes the approach of mapping SPARQL values (that is,
> the XSD datatypes) to the 6 Javascript natural types i.e. xsd:double ->
> JS number - rather than having RDF specific objects (so in style its
> different to "sparql-results+json".
>
> I think that's the right choice of Python and R as well - it's a bridge
> between the worlds, not implementing RDF in language X.
>
> Andy
> >
> > Best,
> >
> > Pavel Mikhailovskii
> > pavel.mikhailovskii@gmail.com (mailto:pavel.mikhailovskii@gmail.com)
> > https://thehyve.nl
> >
> > On Aug 17 2020, at 9:06 am, Maja-Olivia Lenz <le...@fhi-berlin.mpg.de> wrote:
> >> Hi Andy,
> >>
> >> sorry for the delay, I've been on vacation last week.
> >> I want to run a SPARQL query that finds similarities between physical
> >> quantities. To calculate such a similarity I have an external python
> >> function at hand taking several variables as input and returning a
> >> single value as output. So would call something like:
> >>
> >> BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
> >> where the "similarity" is the js-Function that calls the python function.
> >> Maybe there are also other / easier ways than starting a python server?
> >> Jena uses Nashorn, right? It should support exec expressions directly
> >> [1] but gives me an error saying:
> >> "Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
> >> operand but found error"
> >>
> >> In scripting mode there is also $EXEC and $OUT [1] but I don't know how
> >> I can enable scripting mode in my sparql query?
> >>
> >>
> >> Thanks,
> >> Maja
> >>
> >> [1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
> >> [2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
> >>
> >>
> >>
> >> On 8/10/20 18:59, Andy Seaborne wrote:
> >>> Hi Maja,
> >>>
> >>> Before we get into the details, could you say a little more about what
> >>> you are trying to achieve overall? Otherwise we may go off at a tangent.
> >>>
> >>> Andy
> >>>
> >>> On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
> >>>> On 8/6/20 19:23, Andy Seaborne wrote:
> >>>>> You want to make a call out of from SPARQL execution to another
> >>>>> process/another machine?
> >>>>
> >>>> Another process I guess. In my case Jena and Python would be running on
> >>>> the same system.
> >>>>
> >>>>>> Does this make sense/ would that work? Are there
> >>>>>> better ways? Has anyone experience with Javascript/Python integration?
> >>>>>> If I start a SPARQL Query that calls this function very often (say
> >>>>>> 1000/minute) how worse does performance become?
> >>>>>
> >>>>> Possible but if going out to a separate server (same network), that is
> >>>>> not an insignificant cost. Make sure network connects are reused, then a
> >>>>> emote call is less than 1ms of overhead (encode / decode on top if that
> >>>>> will need to be efficient as well).
> >>>>>
> >>>>> That's for custom java code making the call to the python server. I
> >>>>> don't know the overhead for javascript but Jena will run the JS in the
> >>>>> same JVM using Nashorn.
> >>>>
> >>>> So Javascript has some overhead? I've no experience with neither Java or
> >>>> Javascript, I just thought Javascript is easier for someone coming from
> >>>> the Python world plus I've no idea how to integrate python with Java.
> >>>>
> >>>>>
> >>>>> If you could ask for a batch of answers all at once that would reduce
> >>>>> the costs.
> >>>>
> >>>> How would a batch of answers look like in SPARQL? Some concatenation of
> >>>> variables? If I query for some ?x and ?y and give both to python to
> >>>> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
> >>>>
> >>>> Maja
> >>>>
> >>
> >> --
> >> ----------------------------------------------------------------------
> >> Maja-Olivia Lenz
> >> Fritz-Haber-Institut der Max-Planck-Gesellschaft
> >> Faradayweg 4-6
> >> D-14195 Berlin
> >> E-Mail: lenz@fhi-berlin.mpg.de
> >> ---------------------key:1-0.0735-11600-23.05:fhi--------------------
> >>
> >
> >
>
Re: Custom SPARQL functions in Javascript and python
Posted by Andy Seaborne <an...@apache.org>.
GraalVM seems the natural evolution.
GraalVM could also be used to compile a fast-startup version of the
command line tools. It can act as an ahead-of-time (i.e. traditional)
compiler.
On 17/08/2020 12:00, Pavel Mikhailovskii wrote:
> Hi Andy,
>
> By the way, how are you going to tackle the upcoming removal of Nashorn (https://openjdk.java.net/jeps/372)? One option would be to use GraalVM Polyglot API.
> As a quick fix, JSEngine.build could just fall back to Rhino.
> If we agree on the approach, I can start working on that.
> We could even think of supporting other languages than JavaScript, for example Python and R.
That would be interesting
The JS for SPARQL takes the approach of mapping SPARQL values (that is,
the XSD datatypes) to the 6 Javascript natural types i.e. xsd:double ->
JS number - rather than having RDF specific objects (so in style its
different to "sparql-results+json".
I think that's the right choice of Python and R as well - it's a bridge
between the worlds, not implementing RDF in language X.
Andy
>
> Best,
>
> Pavel Mikhailovskii
> pavel.mikhailovskii@gmail.com (mailto:pavel.mikhailovskii@gmail.com)
> https://thehyve.nl
>
> On Aug 17 2020, at 9:06 am, Maja-Olivia Lenz <le...@fhi-berlin.mpg.de> wrote:
>> Hi Andy,
>>
>> sorry for the delay, I've been on vacation last week.
>> I want to run a SPARQL query that finds similarities between physical
>> quantities. To calculate such a similarity I have an external python
>> function at hand taking several variables as input and returning a
>> single value as output. So would call something like:
>>
>> BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
>> where the "similarity" is the js-Function that calls the python function.
>> Maybe there are also other / easier ways than starting a python server?
>> Jena uses Nashorn, right? It should support exec expressions directly
>> [1] but gives me an error saying:
>> "Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
>> operand but found error"
>>
>> In scripting mode there is also $EXEC and $OUT [1] but I don't know how
>> I can enable scripting mode in my sparql query?
>>
>>
>> Thanks,
>> Maja
>>
>> [1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
>> [2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
>>
>>
>>
>> On 8/10/20 18:59, Andy Seaborne wrote:
>>> Hi Maja,
>>>
>>> Before we get into the details, could you say a little more about what
>>> you are trying to achieve overall? Otherwise we may go off at a tangent.
>>>
>>> Andy
>>>
>>> On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
>>>> On 8/6/20 19:23, Andy Seaborne wrote:
>>>>> You want to make a call out of from SPARQL execution to another
>>>>> process/another machine?
>>>>
>>>> Another process I guess. In my case Jena and Python would be running on
>>>> the same system.
>>>>
>>>>>> Does this make sense/ would that work? Are there
>>>>>> better ways? Has anyone experience with Javascript/Python integration?
>>>>>> If I start a SPARQL Query that calls this function very often (say
>>>>>> 1000/minute) how worse does performance become?
>>>>>
>>>>> Possible but if going out to a separate server (same network), that is
>>>>> not an insignificant cost. Make sure network connects are reused, then a
>>>>> emote call is less than 1ms of overhead (encode / decode on top if that
>>>>> will need to be efficient as well).
>>>>>
>>>>> That's for custom java code making the call to the python server. I
>>>>> don't know the overhead for javascript but Jena will run the JS in the
>>>>> same JVM using Nashorn.
>>>>
>>>> So Javascript has some overhead? I've no experience with neither Java or
>>>> Javascript, I just thought Javascript is easier for someone coming from
>>>> the Python world plus I've no idea how to integrate python with Java.
>>>>
>>>>>
>>>>> If you could ask for a batch of answers all at once that would reduce
>>>>> the costs.
>>>>
>>>> How would a batch of answers look like in SPARQL? Some concatenation of
>>>> variables? If I query for some ?x and ?y and give both to python to
>>>> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
>>>>
>>>> Maja
>>>>
>>
>> --
>> ----------------------------------------------------------------------
>> Maja-Olivia Lenz
>> Fritz-Haber-Institut der Max-Planck-Gesellschaft
>> Faradayweg 4-6
>> D-14195 Berlin
>> E-Mail: lenz@fhi-berlin.mpg.de
>> ---------------------key:1-0.0735-11600-23.05:fhi--------------------
>>
>
>
Re: Custom SPARQL functions in Javascript and python
Posted by Pavel Mikhailovskii <pa...@gmail.com>.
Hi Andy,
By the way, how are you going to tackle the upcoming removal of Nashorn (https://openjdk.java.net/jeps/372)? One option would be to use GraalVM Polyglot API.
As a quick fix, JSEngine.build could just fall back to Rhino.
If we agree on the approach, I can start working on that.
We could even think of supporting other languages than JavaScript, for example Python and R.
Best,
Pavel Mikhailovskii
pavel.mikhailovskii@gmail.com (mailto:pavel.mikhailovskii@gmail.com)
https://thehyve.nl
On Aug 17 2020, at 9:06 am, Maja-Olivia Lenz <le...@fhi-berlin.mpg.de> wrote:
> Hi Andy,
>
> sorry for the delay, I've been on vacation last week.
> I want to run a SPARQL query that finds similarities between physical
> quantities. To calculate such a similarity I have an external python
> function at hand taking several variables as input and returning a
> single value as output. So would call something like:
>
> BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
> where the "similarity" is the js-Function that calls the python function.
> Maybe there are also other / easier ways than starting a python server?
> Jena uses Nashorn, right? It should support exec expressions directly
> [1] but gives me an error saying:
> "Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
> operand but found error"
>
> In scripting mode there is also $EXEC and $OUT [1] but I don't know how
> I can enable scripting mode in my sparql query?
>
>
> Thanks,
> Maja
>
> [1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
> [2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
>
>
>
> On 8/10/20 18:59, Andy Seaborne wrote:
> > Hi Maja,
> >
> > Before we get into the details, could you say a little more about what
> > you are trying to achieve overall? Otherwise we may go off at a tangent.
> >
> > Andy
> >
> > On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
> >> On 8/6/20 19:23, Andy Seaborne wrote:
> >>> You want to make a call out of from SPARQL execution to another
> >>> process/another machine?
> >>
> >> Another process I guess. In my case Jena and Python would be running on
> >> the same system.
> >>
> >>>> Does this make sense/ would that work? Are there
> >>>> better ways? Has anyone experience with Javascript/Python integration?
> >>>> If I start a SPARQL Query that calls this function very often (say
> >>>> 1000/minute) how worse does performance become?
> >>>
> >>> Possible but if going out to a separate server (same network), that is
> >>> not an insignificant cost. Make sure network connects are reused, then a
> >>> emote call is less than 1ms of overhead (encode / decode on top if that
> >>> will need to be efficient as well).
> >>>
> >>> That's for custom java code making the call to the python server. I
> >>> don't know the overhead for javascript but Jena will run the JS in the
> >>> same JVM using Nashorn.
> >>
> >> So Javascript has some overhead? I've no experience with neither Java or
> >> Javascript, I just thought Javascript is easier for someone coming from
> >> the Python world plus I've no idea how to integrate python with Java.
> >>
> >>>
> >>> If you could ask for a batch of answers all at once that would reduce
> >>> the costs.
> >>
> >> How would a batch of answers look like in SPARQL? Some concatenation of
> >> variables? If I query for some ?x and ?y and give both to python to
> >> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
> >>
> >> Maja
> >>
>
> --
> ----------------------------------------------------------------------
> Maja-Olivia Lenz
> Fritz-Haber-Institut der Max-Planck-Gesellschaft
> Faradayweg 4-6
> D-14195 Berlin
> E-Mail: lenz@fhi-berlin.mpg.de
> ---------------------key:1-0.0735-11600-23.05:fhi--------------------
>
Re: Custom SPARQL functions in Javascript and python
Posted by Andy Seaborne <an...@apache.org>.
On 26/08/2020 16:07, Andy Seaborne wrote:
...
> While it is using Nashorn, it is doing so via the general javax.script
> package.
>
> My guess is that by default, those extension are not enabled. If
> someone can say how it should be done, or even a pull request to fix it,
> then we can incorporate it.
scripting is enabled by setting a system property on the command line:
java -Dnashorn.args=-scripting ...
Andy
Re: Custom SPARQL functions in Javascript and python
Posted by Andy Seaborne <an...@apache.org>.
On 17/08/2020 08:06, Maja-Olivia Lenz wrote:
> Hi Andy,
>
> sorry for the delay, I've been on vacation last week.
>
> I want to run a SPARQL query that finds similarities between physical
> quantities. To calculate such a similarity I have an external python
> function at hand taking several variables as input and returning a
> single value as output. So would call something like:
>
> BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
>
> where the "similarity" is the js-Function that calls the python function.
>
> Maybe there are also other / easier ways than starting a python server?
>
> Jena uses Nashorn, right?
Yes.
> It should support exec expressions directly
> [1] but gives me an error saying:
> "Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
> operand but found error"
>
> In scripting mode there is also $EXEC and $OUT [1] but I don't know how
> I can enable scripting mode in my sparql query?
While it is using Nashorn, it is doing so via the general javax.script
package.
My guess is that by default, those extension are not enabled. If
someone can say how it should be done, or even a pull request to fix it,
then we can incorporate it.
Jena package:
org.apache.jena.sparql.function.js
But those won't be avilable in GrallVM - or does it have an equivalent?
Andy
>
>
> Thanks,
> Maja
>
> [1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
> [2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
>
>
>
> On 8/10/20 18:59, Andy Seaborne wrote:
>> Hi Maja,
>>
>> Before we get into the details, could you say a little more about what
>> you are trying to achieve overall? Otherwise we may go off at a tangent.
>>
>> Andy
>>
>> On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
>>> On 8/6/20 19:23, Andy Seaborne wrote:
>>>> You want to make a call out of from SPARQL execution to another
>>>> process/another machine?
>>>
>>> Another process I guess. In my case Jena and Python would be running on
>>> the same system.
>>>
>>>>> Does this make sense/ would that work? Are there
>>>>> better ways? Has anyone experience with Javascript/Python integration?
>>>>> If I start a SPARQL Query that calls this function very often (say
>>>>> 1000/minute) how worse does performance become?
>>>>
>>>> Possible but if going out to a separate server (same network), that is
>>>> not an insignificant cost. Make sure network connects are reused, then a
>>>> emote call is less than 1ms of overhead (encode / decode on top if that
>>>> will need to be efficient as well).
>>>>
>>>> That's for custom java code making the call to the python server. I
>>>> don't know the overhead for javascript but Jena will run the JS in the
>>>> same JVM using Nashorn.
>>>
>>> So Javascript has some overhead? I've no experience with neither Java or
>>> Javascript, I just thought Javascript is easier for someone coming from
>>> the Python world plus I've no idea how to integrate python with Java.
>>>
>>>>
>>>> If you could ask for a batch of answers all at once that would reduce
>>>> the costs.
>>>
>>> How would a batch of answers look like in SPARQL? Some concatenation of
>>> variables? If I query for some ?x and ?y and give both to python to
>>> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
>>>
>>> Maja
>>>
>
Re: Custom SPARQL functions in Javascript and python
Posted by Pavel Mikhailovskii <pa...@gmail.com>.
Hi Maja,
Given the error message you get, I suspect that there's a syntax error in your JavaScript.
Pavel Mikhailovskii
pavel.mikhailovskii@gmail.com (mailto:pavel.mikhailovskii@gmail.com)
https://thehyve.nl
On Aug 25 2020, at 10:13 am, Maja-Olivia Lenz <le...@fhi-berlin.mpg.de> wrote:
> Hi all,
>
> sorry to bother you again. Can anyone help with that?
> Thanks,
> Maja
>
> On 8/17/20 09:06, Maja-Olivia Lenz wrote:
> > Hi Andy,
> >
> > sorry for the delay, I've been on vacation last week.
> >
> > I want to run a SPARQL query that finds similarities between physical
> > quantities. To calculate such a similarity I have an external python
> > function at hand taking several variables as input and returning a
> > single value as output. So would call something like:
> >
> > BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
> >
> > where the "similarity" is the js-Function that calls the python function.
> >
> > Maybe there are also other / easier ways than starting a python server?
> >
> > Jena uses Nashorn, right? It should support exec expressions directly
> > [1] but gives me an error saying:
> > "Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
> > operand but found error"
> >
> > In scripting mode there is also $EXEC and $OUT [1] but I don't know how
> > I can enable scripting mode in my sparql query?
> >
> >
> > Thanks,
> > Maja
> >
> > [1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
> > [2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
> >
> >
> >
> > On 8/10/20 18:59, Andy Seaborne wrote:
> >> Hi Maja,
> >>
> >> Before we get into the details, could you say a little more about what
> >> you are trying to achieve overall? Otherwise we may go off at a tangent.
> >>
> >> Andy
> >>
> >> On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
> >>> On 8/6/20 19:23, Andy Seaborne wrote:
> >>>> You want to make a call out of from SPARQL execution to another
> >>>> process/another machine?
> >>>
> >>> Another process I guess. In my case Jena and Python would be running on
> >>> the same system.
> >>>
> >>>>> Does this make sense/ would that work? Are there
> >>>>> better ways? Has anyone experience with Javascript/Python integration?
> >>>>> If I start a SPARQL Query that calls this function very often (say
> >>>>> 1000/minute) how worse does performance become?
> >>>>
> >>>> Possible but if going out to a separate server (same network), that is
> >>>> not an insignificant cost. Make sure network connects are reused, then a
> >>>> emote call is less than 1ms of overhead (encode / decode on top if that
> >>>> will need to be efficient as well).
> >>>>
> >>>> That's for custom java code making the call to the python server. I
> >>>> don't know the overhead for javascript but Jena will run the JS in the
> >>>> same JVM using Nashorn.
> >>>
> >>> So Javascript has some overhead? I've no experience with neither Java or
> >>> Javascript, I just thought Javascript is easier for someone coming from
> >>> the Python world plus I've no idea how to integrate python with Java.
> >>>
> >>>>
> >>>> If you could ask for a batch of answers all at once that would reduce
> >>>> the costs.
> >>>
> >>> How would a batch of answers look like in SPARQL? Some concatenation of
> >>> variables? If I query for some ?x and ?y and give both to python to
> >>> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
> >>>
> >>> Maja
> >>>
> >
>
> --
> ----------------------------------------------------------------------
> Maja-Olivia Lenz
> Fritz-Haber-Institut der Max-Planck-Gesellschaft
> Faradayweg 4-6
> D-14195 Berlin
> E-Mail: lenz@fhi-berlin.mpg.de
> ---------------------key:1-0.0735-11600-23.05:fhi--------------------
>
Re: Custom SPARQL functions in Javascript and python
Posted by Maja-Olivia Lenz <le...@fhi-berlin.mpg.de>.
Hi all,
sorry to bother you again. Can anyone help with that?
Thanks,
Maja
On 8/17/20 09:06, Maja-Olivia Lenz wrote:
> Hi Andy,
>
> sorry for the delay, I've been on vacation last week.
>
> I want to run a SPARQL query that finds similarities between physical
> quantities. To calculate such a similarity I have an external python
> function at hand taking several variables as input and returning a
> single value as output. So would call something like:
>
> BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
>
> where the "similarity" is the js-Function that calls the python function.
>
> Maybe there are also other / easier ways than starting a python server?
>
> Jena uses Nashorn, right? It should support exec expressions directly
> [1] but gives me an error saying:
> "Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
> operand but found error"
>
> In scripting mode there is also $EXEC and $OUT [1] but I don't know how
> I can enable scripting mode in my sparql query?
>
>
> Thanks,
> Maja
>
> [1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
> [2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
>
>
>
> On 8/10/20 18:59, Andy Seaborne wrote:
>> Hi Maja,
>>
>> Before we get into the details, could you say a little more about what
>> you are trying to achieve overall? Otherwise we may go off at a tangent.
>>
>> Andy
>>
>> On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
>>> On 8/6/20 19:23, Andy Seaborne wrote:
>>>> You want to make a call out of from SPARQL execution to another
>>>> process/another machine?
>>>
>>> Another process I guess. In my case Jena and Python would be running on
>>> the same system.
>>>
>>>>> Does this make sense/ would that work? Are there
>>>>> better ways? Has anyone experience with Javascript/Python integration?
>>>>> If I start a SPARQL Query that calls this function very often (say
>>>>> 1000/minute) how worse does performance become?
>>>>
>>>> Possible but if going out to a separate server (same network), that is
>>>> not an insignificant cost. Make sure network connects are reused, then a
>>>> emote call is less than 1ms of overhead (encode / decode on top if that
>>>> will need to be efficient as well).
>>>>
>>>> That's for custom java code making the call to the python server. I
>>>> don't know the overhead for javascript but Jena will run the JS in the
>>>> same JVM using Nashorn.
>>>
>>> So Javascript has some overhead? I've no experience with neither Java or
>>> Javascript, I just thought Javascript is easier for someone coming from
>>> the Python world plus I've no idea how to integrate python with Java.
>>>
>>>>
>>>> If you could ask for a batch of answers all at once that would reduce
>>>> the costs.
>>>
>>> How would a batch of answers look like in SPARQL? Some concatenation of
>>> variables? If I query for some ?x and ?y and give both to python to
>>> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
>>>
>>> Maja
>>>
>
--
----------------------------------------------------------------------
Maja-Olivia Lenz
Fritz-Haber-Institut der Max-Planck-Gesellschaft
Faradayweg 4-6
D-14195 Berlin
E-Mail: lenz@fhi-berlin.mpg.de
---------------------key:1-0.0735-11600-23.05:fhi--------------------
Re: Custom SPARQL functions in Javascript and python
Posted by Maja-Olivia Lenz <le...@fhi-berlin.mpg.de>.
Hi Andy,
sorry for the delay, I've been on vacation last week.
I want to run a SPARQL query that finds similarities between physical
quantities. To calculate such a similarity I have an external python
function at hand taking several variables as input and returning a
single value as output. So would call something like:
BIND(js:similarity(?x1, ?y1, ?x2, ?y2) as ?sim)
where the "similarity" is the js-Function that calls the python function.
Maybe there are also other / easier ways than starting a python server?
Jena uses Nashorn, right? It should support exec expressions directly
[1] but gives me an error saying:
"Caused by: javax.script.ScriptException: <eval>:6:17 Expected an
operand but found error"
In scripting mode there is also $EXEC and $OUT [1] but I don't know how
I can enable scripting mode in my sparql query?
Thanks,
Maja
[1] https://www.n-k.de/riding-the-nashorn/#_back_quote_exec_expressions
[2] https://www.n-k.de/riding-the-nashorn/#_scripting_mode_extension_objects
On 8/10/20 18:59, Andy Seaborne wrote:
> Hi Maja,
>
> Before we get into the details, could you say a little more about what
> you are trying to achieve overall? Otherwise we may go off at a tangent.
>
> Andy
>
> On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
>> On 8/6/20 19:23, Andy Seaborne wrote:
>>> You want to make a call out of from SPARQL execution to another
>>> process/another machine?
>>
>> Another process I guess. In my case Jena and Python would be running on
>> the same system.
>>
>>>> Does this make sense/ would that work? Are there
>>>> better ways? Has anyone experience with Javascript/Python integration?
>>>> If I start a SPARQL Query that calls this function very often (say
>>>> 1000/minute) how worse does performance become?
>>>
>>> Possible but if going out to a separate server (same network), that is
>>> not an insignificant cost. Make sure network connects are reused, then a
>>> emote call is less than 1ms of overhead (encode / decode on top if that
>>> will need to be efficient as well).
>>>
>>> That's for custom java code making the call to the python server. I
>>> don't know the overhead for javascript but Jena will run the JS in the
>>> same JVM using Nashorn.
>>
>> So Javascript has some overhead? I've no experience with neither Java or
>> Javascript, I just thought Javascript is easier for someone coming from
>> the Python world plus I've no idea how to integrate python with Java.
>>
>>>
>>> If you could ask for a batch of answers all at once that would reduce
>>> the costs.
>>
>> How would a batch of answers look like in SPARQL? Some concatenation of
>> variables? If I query for some ?x and ?y and give both to python to
>> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
>>
>> Maja
>>
--
----------------------------------------------------------------------
Maja-Olivia Lenz
Fritz-Haber-Institut der Max-Planck-Gesellschaft
Faradayweg 4-6
D-14195 Berlin
E-Mail: lenz@fhi-berlin.mpg.de
---------------------key:1-0.0735-11600-23.05:fhi--------------------
Re: Custom SPARQL functions in Javascript and python
Posted by Andy Seaborne <an...@apache.org>.
Hi Maja,
Before we get into the details, could you say a little more about what
you are trying to achieve overall? Otherwise we may go off at a tangent.
Andy
On 07/08/2020 08:14, Maja-Olivia Lenz wrote:
> On 8/6/20 19:23, Andy Seaborne wrote:
>> You want to make a call out of from SPARQL execution to another
>> process/another machine?
>
> Another process I guess. In my case Jena and Python would be running on
> the same system.
>
>>> Does this make sense/ would that work? Are there
>>> better ways? Has anyone experience with Javascript/Python integration?
>>> If I start a SPARQL Query that calls this function very often (say
>>> 1000/minute) how worse does performance become?
>>
>> Possible but if going out to a separate server (same network), that is
>> not an insignificant cost. Make sure network connects are reused, then a
>> emote call is less than 1ms of overhead (encode / decode on top if that
>> will need to be efficient as well).
>>
>> That's for custom java code making the call to the python server. I
>> don't know the overhead for javascript but Jena will run the JS in the
>> same JVM using Nashorn.
>
> So Javascript has some overhead? I've no experience with neither Java or
> Javascript, I just thought Javascript is easier for someone coming from
> the Python world plus I've no idea how to integrate python with Java.
>
>>
>> If you could ask for a batch of answers all at once that would reduce
>> the costs.
>
> How would a batch of answers look like in SPARQL? Some concatenation of
> variables? If I query for some ?x and ?y and give both to python to
> return ?z, then I don't see how I can aggregate all ?x ?y pairs.
>
> Maja
>
Re: Custom SPARQL functions in Javascript and python
Posted by Maja-Olivia Lenz <le...@fhi-berlin.mpg.de>.
On 8/6/20 19:23, Andy Seaborne wrote:
> You want to make a call out of from SPARQL execution to another
> process/another machine?
Another process I guess. In my case Jena and Python would be running on
the same system.
>> Does this make sense/ would that work? Are there
>> better ways? Has anyone experience with Javascript/Python integration?
>> If I start a SPARQL Query that calls this function very often (say
>> 1000/minute) how worse does performance become?
>
> Possible but if going out to a separate server (same network), that is
> not an insignificant cost. Make sure network connects are reused, then a
> emote call is less than 1ms of overhead (encode / decode on top if that
> will need to be efficient as well).
>
> That's for custom java code making the call to the python server. I
> don't know the overhead for javascript but Jena will run the JS in the
> same JVM using Nashorn.
So Javascript has some overhead? I've no experience with neither Java or
Javascript, I just thought Javascript is easier for someone coming from
the Python world plus I've no idea how to integrate python with Java.
>
> If you could ask for a batch of answers all at once that would reduce
> the costs.
How would a batch of answers look like in SPARQL? Some concatenation of
variables? If I query for some ?x and ?y and give both to python to
return ?z, then I don't see how I can aggregate all ?x ?y pairs.
Maja
Re: Custom SPARQL functions in Javascript and python
Posted by Andy Seaborne <an...@apache.org>.
On 06/08/2020 15:20, Maja-Olivia Lenz wrote:
> Hi all,
>
> I have read that it's possible in Jena to write custom functions for
> SPARQL in Java or Javascript.
Yes - for Java, add implementation classes to the classpath and either
call as <java:...> or register explitly via
https://jena.apache.org/documentation/notes/system-initialization
https://jena.apache.org/documentation/query/javascript-functions
> Is it possible to interface this with
> existing python functions? My idea is to start a python server (e.g.
> flask) and send post requests through Javascript to obtain the
> quantities I need.
You want to make a call out of from SPARQL execution to another
process/another machine?
> Does this make sense/ would that work? Are there
> better ways? Has anyone experience with Javascript/Python integration?
> If I start a SPARQL Query that calls this function very often (say
> 1000/minute) how worse does performance become?
Possible but if going out to a separate server (same network), that is
not an insignificant cost. Make sure network connects are reused, then a
emote call is less than 1ms of overhead (encode / decode on top if that
will need to be efficient as well).
That's for custom java code making the call to the python server. I
don't know the overhead for javascript but Jena will run the JS in the
same JVM using Nashorn.
If you could ask for a batch of answers all at once that would reduce
the costs.
> I don't know whether it matters: So far I've only used ARQ on rdf files
> or a local tdb database, however, in the future I'd also like to use Fuseki.
From the point of view of this question, it is much the same; with
Fuseki you can run with additional classes.
>
> Thanks in advance,
> Maja
Andy