You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Dimitris Spanos <di...@gmail.com> on 2020/08/15 10:59:06 UTC
Custom QueryEngine is not picked up by QueryExecutionFactory in
presence of FROM/FROM NAMED clause
Hi all,
I am trying to implement a custom QueryEngine that would handle query
optimization and evaluation for a custom Dataset implementation, as
mentioned in the documentation (
https://jena.apache.org/documentation/query/arq-query-eval.html#custom-query-engines
).
Following the lines in the example in
src-examples/arq.examples.engine.MyQueryEngine, I implemented a similar
MyQueryEngine and MyQueryEngineFactory, with the accept() implementation
looking like the following:
public boolean accept(Query query, DatasetGraph dataset, Context context) {
return dataset instanceof MyDatasetGraph;
}
where MyDatasetGraph is a DatasetGraph implementation, backing the
custom Dataset implementation, for which I want to use MyQueryEngine.
While this works fine for a simple query without any FROM/FROM NAMED
clause, the above accept() method does not work anymore when the query
includes a dataset specification in a FROM/FROM NAMED clause.
This happens because in the query processing phase occurring in
SPARQL_QueryDataset, MyDatasetGraph gets wrapped in a
DynamicDatasetGraph, which does not seem to have any public methods to
get the wrapped graph.
Q1: Is there a way to determine whether a DynamicDatasetGraph wraps an
instance of MyDatasetGraph? If not, could I somehow use Context to
determine the type of Dataset/DatasetGraph being queried?
Q2: I noticed that this behaviour occurs only in SPARQL_QueryDataset, which
is the default ActionService for the "query" Operation in Fuseki. It could
be that using SPARQL_QueryGeneral for the Query operation might solve the
problem for me, but can this be configured in Fuseki configuration? I could
not find anything relevant in
https://jena.apache.org/documentation/fuseki2/fuseki-config-endpoint.html.
Q3: By digging some more, I saw that SPARQL_QueryGeneral is registered as
an ActionService for the "query" operation when Fuseki starts with either
of the --sparqler or --general arguments. Nevertheless, I should mention
that I have a strong preference in starting Fuseki as a WAR with just a
configuration template for my new dataset type. Can I pass command line
arguments or even start Fuseki programmatically while still running it as a
Web Application?
Thanks in advance,
Dimitris
Re: Custom QueryEngine is not picked up by QueryExecutionFactory in
presence of FROM/FROM NAMED clause
Posted by Andy Seaborne <an...@apache.org>.
On 16/08/2020 10:09, Dimitris Spanos wrote:
> Hi Andy,
>
> Thank you for the extremely useful information.
>
> Indeed, I want to pass the default graph and named graph URIs to MyQueryEngine, so I went for your advice and extended SPARQLQueryProcessor accordingly to store this information in the context of MyDatasetGraph.
>
> I could have retrieved the same information from DynamicDatasetGraph (via getWrapped() as you said), but I think the current solution leads to cleaner code.
>
> Conveniently, in my Jena subsystem "init" class, I also registered the above handler to the Query operation with FusekiExt.registerOperation(), avoiding any changes to Fuseki Core or configuring/starting the server programmatically.
I had completely forgotten about FusekiExt!
Andy
>
> Cheers,
> Dimitris
>
> On 2020/08/15 13:08:21, Andy Seaborne <an...@apache.org> wrote:
>>
>>
>> On 15/08/2020 11:59, Dimitris Spanos wrote:
>>> Hi all,
>>>
>>> I am trying to implement a custom QueryEngine that would handle query
>>> optimization and evaluation for a custom Dataset implementation, as
>>> mentioned in the documentation (
>>> https://jena.apache.org/documentation/query/arq-query-eval.html#custom-query-engines
>>> ).
>>>
>>> Following the lines in the example in
>>> src-examples/arq.examples.engine.MyQueryEngine, I implemented a similar
>>> MyQueryEngine and MyQueryEngineFactory, with the accept() implementation
>>> looking like the following:
>>>
>>> public boolean accept(Query query, DatasetGraph dataset, Context context) {
>>> return dataset instanceof MyDatasetGraph;
>>> }
>>>
>>> where MyDatasetGraph is a DatasetGraph implementation, backing the
>>> custom Dataset implementation, for which I want to use MyQueryEngine.
>>>
>>> While this works fine for a simple query without any FROM/FROM NAMED
>>> clause, the above accept() method does not work anymore when the query
>>> includes a dataset specification in a FROM/FROM NAMED clause.
>>
>> Just to check - what effect are you looking for by using FROM/FROM
>> NAMED? just passthru to the query engine?
>>
>>> This happens because in the query processing phase occurring in
>>> SPARQL_QueryDataset, MyDatasetGraph gets wrapped in a
>>> DynamicDatasetGraph, which does not seem to have any public methods to
>>> get the wrapped graph.
>>
>> You can add your own query processor if the provided one does not suit
>> your needs.
>>
>> Override "decideDataset" to return as if no FROM/FROM NAMED were present.
>>
>> org.apache.jena.fuseki.main.examples.ExtendFuseki_AddService_1
>>
>> (for Fuseki main but the mechanism underneath is part of core Fuseki -
>> find the operation registry registry and add your own.)
>>
>> (Or replace the standard registration - that will then work with
>> dispatch by content-type - there can be only one process for each
>> content type. It is also how dispatch on "/dataset?query" (no service
>> name) works - again, there can be only one processor for "?query" at an
>> endpoint.)
>>
>>
>>> Q1: Is there a way to determine whether a DynamicDatasetGraph wraps an
>>> instance of MyDatasetGraph? If not, could I somehow use Context to
>>> determine the type of Dataset/DatasetGraph being queried?
>>
>> public static class DynamicDatasetGraph
>> extends DatasetGraphReadOnly
>> implements DatasetGraphWrapperView
>>
>> DatasetGraphReadOnly extends DatasetGraphWrapper
>>
>> This should work:
>>
>> DatasetGraphWrapper.getWrapped()
>>
>>> Q2: I noticed that this behaviour occurs only in SPARQL_QueryDataset, which
>>> is the default ActionService for the "query" Operation in Fuseki. It could
>>> be that using SPARQL_QueryGeneral for the Query operation might solve the
>>> problem for me, but can this be configured in Fuseki configuration? I could
>>> not find anything relevant in
>>> https://jena.apache.org/documentation/fuseki2/fuseki-config-endpoint.html.
>>
>> SPARQL_QueryGeneral is a standalone servlet to go with an HTML form and
>> there is no dataset behind it.
>>
>> e.g. http://www.sparql.org/sparql.html
>>
>> The form has to provide the data - e.g. a query with VALUES clause - a
>> FROM would load from the web.
>>
>> It can't be graph from the dataset because there isn't one (well, there
>> is a fake immutable empty one)
>>
>>> Q3: By digging some more, I saw that SPARQL_QueryGeneral is registered as
>>> an ActionService for the "query" operation when Fuseki starts with either
>>> of the --sparqler or --general arguments. Nevertheless, I should mention
>>> that I have a strong preference in starting Fuseki as a WAR with just a
>>> configuration template for my new dataset type. Can I pass command line
>>> arguments or even start Fuseki programmatically while still running it as a
>>> Web Application?
>>
>> If you mean so it can be dropped into a web container? You an start
>> Fuseki (main) inside an application that is inside a WAR file. Depeding
>> on webapp container, ports may be complicated.
>>
>> jena-fuseki-war is a drop in WAR file.
>>
>> jena-fuseki-full jar is Fuseki packaged with Jetty and is a web application.
>>
>> jena-fuseki-server is a combined, but non-war version.
>>
>>
>>>
>>> Thanks in advance,
>>> Dimitris
>>>
>>
Re: Custom QueryEngine is not picked up by QueryExecutionFactory in presence of FROM/FROM NAMED clause
Posted by Dimitris Spanos <di...@gmail.com>.
Hi Andy,
Thank you for the extremely useful information.
Indeed, I want to pass the default graph and named graph URIs to MyQueryEngine, so I went for your advice and extended SPARQLQueryProcessor accordingly to store this information in the context of MyDatasetGraph.
I could have retrieved the same information from DynamicDatasetGraph (via getWrapped() as you said), but I think the current solution leads to cleaner code.
Conveniently, in my Jena subsystem "init" class, I also registered the above handler to the Query operation with FusekiExt.registerOperation(), avoiding any changes to Fuseki Core or configuring/starting the server programmatically.
Cheers,
Dimitris
On 2020/08/15 13:08:21, Andy Seaborne <an...@apache.org> wrote:
>
>
> On 15/08/2020 11:59, Dimitris Spanos wrote:
> > Hi all,
> >
> > I am trying to implement a custom QueryEngine that would handle query
> > optimization and evaluation for a custom Dataset implementation, as
> > mentioned in the documentation (
> > https://jena.apache.org/documentation/query/arq-query-eval.html#custom-query-engines
> > ).
> >
> > Following the lines in the example in
> > src-examples/arq.examples.engine.MyQueryEngine, I implemented a similar
> > MyQueryEngine and MyQueryEngineFactory, with the accept() implementation
> > looking like the following:
> >
> > public boolean accept(Query query, DatasetGraph dataset, Context context) {
> > return dataset instanceof MyDatasetGraph;
> > }
> >
> > where MyDatasetGraph is a DatasetGraph implementation, backing the
> > custom Dataset implementation, for which I want to use MyQueryEngine.
> >
> > While this works fine for a simple query without any FROM/FROM NAMED
> > clause, the above accept() method does not work anymore when the query
> > includes a dataset specification in a FROM/FROM NAMED clause.
>
> Just to check - what effect are you looking for by using FROM/FROM
> NAMED? just passthru to the query engine?
>
> > This happens because in the query processing phase occurring in
> > SPARQL_QueryDataset, MyDatasetGraph gets wrapped in a
> > DynamicDatasetGraph, which does not seem to have any public methods to
> > get the wrapped graph.
>
> You can add your own query processor if the provided one does not suit
> your needs.
>
> Override "decideDataset" to return as if no FROM/FROM NAMED were present.
>
> org.apache.jena.fuseki.main.examples.ExtendFuseki_AddService_1
>
> (for Fuseki main but the mechanism underneath is part of core Fuseki -
> find the operation registry registry and add your own.)
>
> (Or replace the standard registration - that will then work with
> dispatch by content-type - there can be only one process for each
> content type. It is also how dispatch on "/dataset?query" (no service
> name) works - again, there can be only one processor for "?query" at an
> endpoint.)
>
>
> > Q1: Is there a way to determine whether a DynamicDatasetGraph wraps an
> > instance of MyDatasetGraph? If not, could I somehow use Context to
> > determine the type of Dataset/DatasetGraph being queried?
>
> public static class DynamicDatasetGraph
> extends DatasetGraphReadOnly
> implements DatasetGraphWrapperView
>
> DatasetGraphReadOnly extends DatasetGraphWrapper
>
> This should work:
>
> DatasetGraphWrapper.getWrapped()
>
> > Q2: I noticed that this behaviour occurs only in SPARQL_QueryDataset, which
> > is the default ActionService for the "query" Operation in Fuseki. It could
> > be that using SPARQL_QueryGeneral for the Query operation might solve the
> > problem for me, but can this be configured in Fuseki configuration? I could
> > not find anything relevant in
> > https://jena.apache.org/documentation/fuseki2/fuseki-config-endpoint.html.
>
> SPARQL_QueryGeneral is a standalone servlet to go with an HTML form and
> there is no dataset behind it.
>
> e.g. http://www.sparql.org/sparql.html
>
> The form has to provide the data - e.g. a query with VALUES clause - a
> FROM would load from the web.
>
> It can't be graph from the dataset because there isn't one (well, there
> is a fake immutable empty one)
>
> > Q3: By digging some more, I saw that SPARQL_QueryGeneral is registered as
> > an ActionService for the "query" operation when Fuseki starts with either
> > of the --sparqler or --general arguments. Nevertheless, I should mention
> > that I have a strong preference in starting Fuseki as a WAR with just a
> > configuration template for my new dataset type. Can I pass command line
> > arguments or even start Fuseki programmatically while still running it as a
> > Web Application?
>
> If you mean so it can be dropped into a web container? You an start
> Fuseki (main) inside an application that is inside a WAR file. Depeding
> on webapp container, ports may be complicated.
>
> jena-fuseki-war is a drop in WAR file.
>
> jena-fuseki-full jar is Fuseki packaged with Jetty and is a web application.
>
> jena-fuseki-server is a combined, but non-war version.
>
>
> >
> > Thanks in advance,
> > Dimitris
> >
>
Re: Custom QueryEngine is not picked up by QueryExecutionFactory in
presence of FROM/FROM NAMED clause
Posted by Andy Seaborne <an...@apache.org>.
On 15/08/2020 11:59, Dimitris Spanos wrote:
> Hi all,
>
> I am trying to implement a custom QueryEngine that would handle query
> optimization and evaluation for a custom Dataset implementation, as
> mentioned in the documentation (
> https://jena.apache.org/documentation/query/arq-query-eval.html#custom-query-engines
> ).
>
> Following the lines in the example in
> src-examples/arq.examples.engine.MyQueryEngine, I implemented a similar
> MyQueryEngine and MyQueryEngineFactory, with the accept() implementation
> looking like the following:
>
> public boolean accept(Query query, DatasetGraph dataset, Context context) {
> return dataset instanceof MyDatasetGraph;
> }
>
> where MyDatasetGraph is a DatasetGraph implementation, backing the
> custom Dataset implementation, for which I want to use MyQueryEngine.
>
> While this works fine for a simple query without any FROM/FROM NAMED
> clause, the above accept() method does not work anymore when the query
> includes a dataset specification in a FROM/FROM NAMED clause.
Just to check - what effect are you looking for by using FROM/FROM
NAMED? just passthru to the query engine?
> This happens because in the query processing phase occurring in
> SPARQL_QueryDataset, MyDatasetGraph gets wrapped in a
> DynamicDatasetGraph, which does not seem to have any public methods to
> get the wrapped graph.
You can add your own query processor if the provided one does not suit
your needs.
Override "decideDataset" to return as if no FROM/FROM NAMED were present.
org.apache.jena.fuseki.main.examples.ExtendFuseki_AddService_1
(for Fuseki main but the mechanism underneath is part of core Fuseki -
find the operation registry registry and add your own.)
(Or replace the standard registration - that will then work with
dispatch by content-type - there can be only one process for each
content type. It is also how dispatch on "/dataset?query" (no service
name) works - again, there can be only one processor for "?query" at an
endpoint.)
> Q1: Is there a way to determine whether a DynamicDatasetGraph wraps an
> instance of MyDatasetGraph? If not, could I somehow use Context to
> determine the type of Dataset/DatasetGraph being queried?
public static class DynamicDatasetGraph
extends DatasetGraphReadOnly
implements DatasetGraphWrapperView
DatasetGraphReadOnly extends DatasetGraphWrapper
This should work:
DatasetGraphWrapper.getWrapped()
> Q2: I noticed that this behaviour occurs only in SPARQL_QueryDataset, which
> is the default ActionService for the "query" Operation in Fuseki. It could
> be that using SPARQL_QueryGeneral for the Query operation might solve the
> problem for me, but can this be configured in Fuseki configuration? I could
> not find anything relevant in
> https://jena.apache.org/documentation/fuseki2/fuseki-config-endpoint.html.
SPARQL_QueryGeneral is a standalone servlet to go with an HTML form and
there is no dataset behind it.
e.g. http://www.sparql.org/sparql.html
The form has to provide the data - e.g. a query with VALUES clause - a
FROM would load from the web.
It can't be graph from the dataset because there isn't one (well, there
is a fake immutable empty one)
> Q3: By digging some more, I saw that SPARQL_QueryGeneral is registered as
> an ActionService for the "query" operation when Fuseki starts with either
> of the --sparqler or --general arguments. Nevertheless, I should mention
> that I have a strong preference in starting Fuseki as a WAR with just a
> configuration template for my new dataset type. Can I pass command line
> arguments or even start Fuseki programmatically while still running it as a
> Web Application?
If you mean so it can be dropped into a web container? You an start
Fuseki (main) inside an application that is inside a WAR file. Depeding
on webapp container, ports may be complicated.
jena-fuseki-war is a drop in WAR file.
jena-fuseki-full jar is Fuseki packaged with Jetty and is a web application.
jena-fuseki-server is a combined, but non-war version.
>
> Thanks in advance,
> Dimitris
>