You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Paolo Castagna <ca...@googlemail.com> on 2012/01/13 16:45:30 UTC
Using SERVICE with parameters: HttpException: -1 URL already has
a query string
Hi,
today, I colleague of mine was trying to use SERVICE in a SPARQL query against a
SPARQL endpoint which requires some additional parameters (i.e. for example, an
apikey=... for authentication).
So, he has this in his query:
SERVICE <http://.../sparql?apikey=...> {
[...]
}
This is the exception he gets (for example, running the query via arq.query
command line):
HttpException: -1 URL already has a query string (http://.../sparql?apikey=...)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.init(HttpQuery.java:100)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.<init>(HttpQuery.java:80)
at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:68)
at
com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
at com.hp.hpl.jena.sparql.resultset.ResultSetMem.<init>(ResultSetMem.java:95)
at com.hp.hpl.jena.sparql.resultset.TextOutput.write(TextOutput.java:147)
at com.hp.hpl.jena.sparql.resultset.TextOutput.write(TextOutput.java:130)
at com.hp.hpl.jena.sparql.resultset.TextOutput.write(TextOutput.java:118)
at com.hp.hpl.jena.sparql.resultset.TextOutput.format(TextOutput.java:65)
at com.hp.hpl.jena.query.ResultSetFormatter.out(ResultSetFormatter.java:134)
at
com.hp.hpl.jena.sparql.util.QueryExecUtils.outputResultSet(QueryExecUtils.java:169)
at
com.hp.hpl.jena.sparql.util.QueryExecUtils.doSelectQuery(QueryExecUtils.java:211)
at com.hp.hpl.jena.sparql.util.QueryExecUtils.executeQuery(QueryExecUtils.java:75)
at arq.query.queryExec(query.java:186)
at arq.query.exec(query.java:145)
at arq.cmdline.CmdMain.mainMethod(CmdMain.java:97)
at arq.cmdline.CmdMain.mainRun(CmdMain.java:59)
at arq.cmdline.CmdMain.mainRun(CmdMain.java:46)
at arq.query.main(query.java:65)
Would it be possible to relax the check in HttpQuery.java?
if ( serviceURL.indexOf('?') >= 0 )
throw new QueryExceptionHTTP(-1, "URL already has a query string
("+serviceURL+")") ;
Or, the SPARQL specs explicitly forbid that... probably here, isn't it?
http://www.w3.org/TR/sparql11-query/#rIRI_REF
Luckily, QueryEngineHTTP has an addParam(...) method. However, I did not find
a way do go around this within a SPARQL query using SERVICE.
Any comment/suggestion?
Thanks in advance,
Paolo
Re: JENA-195 : a small point of project policy on other-system specific
code.
Posted by Andy Seaborne <an...@apache.org>.
On 19/01/12 21:26, Paolo Castagna wrote:
> Andy Seaborne wrote:
>> There is already a DBpedia example so we have included a
>> service-specific examples before. DBpedia is used by students and in
>> student projects so we do get a stream of questions about using it,
>> especially as it can be peculiar in its timeout behaviour.
>
> Not very useful, but it shows users that you can specify query parameters
> in the SERVICE<IRI>:
>
> String queryString =
> "SELECT * WHERE { " +
> " SERVICE<http://dbpedia-live.openlinksw.com/sparql?timeout=10000> { " +
> " SELECT DISTINCT ?company where {?company a<http://dbpedia.org/ontology/Company>} LIMIT 20" +
> " }" +
> "}" ;
> Query query = QueryFactory.create(queryString) ;
> QueryExecution qexec = QueryExecutionFactory.create(query, ModelFactory.createDefaultModel()) ;
> try {
> ResultSet rs = qexec.execSelect() ;
> ResultSetFormatter.out(System.out, rs, query) ;
> } finally {
> qexec.close() ;
> }
>
> Maybe something like this is better (and it should not increase the number
> of questions.
>
> I do not know if the DBPedia service supports other query parameters that
> could make the example more useful/interesting.
>
> Paolo
Good use of DBpedia.
I think it's good to illustrate this with a use of setting the param via
the context (so the timeout is not in the query itself and can be
adjusted more easily). We can use such example code to test the
proposed patch.
Andy
Re: JENA-195 : a small point of project policy on other-system specific
code.
Posted by Paolo Castagna <ca...@googlemail.com>.
Andy Seaborne wrote:
> There is already a DBpedia example so we have included a
> service-specific examples before. DBpedia is used by students and in
> student projects so we do get a stream of questions about using it,
> especially as it can be peculiar in its timeout behaviour.
Not very useful, but it shows users that you can specify query parameters
in the SERVICE <IRI>:
String queryString =
"SELECT * WHERE { " +
" SERVICE <http://dbpedia-live.openlinksw.com/sparql?timeout=10000> { " +
" SELECT DISTINCT ?company where {?company a <http://dbpedia.org/ontology/Company>} LIMIT 20" +
" }" +
"}" ;
Query query = QueryFactory.create(queryString) ;
QueryExecution qexec = QueryExecutionFactory.create(query, ModelFactory.createDefaultModel()) ;
try {
ResultSet rs = qexec.execSelect() ;
ResultSetFormatter.out(System.out, rs, query) ;
} finally {
qexec.close() ;
}
Maybe something like this is better (and it should not increase the number
of questions.
I do not know if the DBPedia service supports other query parameters that
could make the example more useful/interesting.
Paolo
Re: JENA-195 : a small point of project policy on other-system specific
code.
Posted by Paolo Castagna <ca...@googlemail.com>.
Hi
Andy Seaborne wrote:
> The patch for JENA-195 from Paolo includes a Kasabi-specific example in
> the ARQ examples. The example is both specific to Kasabi and to a
> particular Kasabi dataset.
... and it requires an API key (in an environment variable), with hindsight
not a good idea. :-) I removed it and posted a new patch without that example.
Paolo
>
> There is already a DBpedia example so we have included a
> service-specific examples before. DBpedia is used by students and in
> student projects so we do get a stream of questions about using it,
> especially as it can be peculiar in its timeout behaviour.
>
> However, this case is different because Kasabi requires registration to
> obtain an API key and the example does not work without such
> registration. The key is required to be in environment variable
> KASABI_API_KEY, which is the way all Kasabi client APIs deal with the
> API key.
>
> A committer can not test the example unless they register with Kasabi.
> This, to me, is the critical difference here.
>
> Personally, I think such examples should go on the Kasabi site and not
> in the jena-arq codebase, and eventually Jena examples area which Ian's
> been setting up.
>
> If the majority feel this is acceptable, then I will not block it being
> included.
>
> Andy
>
> -------- Original Message --------
> Subject: Re: Using SERVICE with parameters: HttpException: -1 URL
> already has a query string
> Date: Thu, 19 Jan 2012 08:40:41 +0000
> From: Paolo Castagna <ca...@googlemail.com>
> Reply-To: jena-users@incubator.apache.org
> To: jena-users@incubator.apache.org
>
> Andy Seaborne wrote:
>>> My colleague was using arq.query command line, so he was trying to
>>> specify
>>> additional query parameters in his SPARQL queries writing:
>>>
>>> SELECT ... WHERE {
>>> SERVICE<...?name1=value1> { ... }
>>> SERVICE<...?name2=value2> { ... }
>>> }
>>>
>>> I don't think this is allowed by the SPARQL grammar. Is it?
>>
>> Have you tried the online query validator?
>>
>> http://www.sparql.org/query-validator.html
>>
>
> Hi Andy,
> at first, I did not try the validator (I should have).
> I looked at the SPARQL grammar [1] instead.
>
> In particular, the thing I am still not sure is if <http://.../?name=value>
> is ok as IRI_REF [2].
>
> The on-line validator has no problems with SPARQL queries containing:
> SERVICE <http://.../?name=value>. Good.
>
> ARQ currently throws an exception if someone attempts to use query
> parameters in a SERVICE <IRI>.
>
> We now have a patch [3] which, instead of throwing an exception when '?'
> is found in the SERVICE <IRI>, keep the query string parameters and send
> them to the remote SPARQL endpoint as is. Here is the relevant bit:
>
> Index: src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
> ===================================================================
> --- src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
> (revision
> 1232106)
> +++ src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
> (working copy)
> @@ -68,6 +68,7 @@
> String responseMessage = null ;
> boolean forcePOST = false ;
> String queryString = null ;
> + boolean serviceParams = false ;
>
> //static final String ENC_UTF8 = "UTF-8" ;
>
> @@ -97,7 +98,7 @@
> log.trace("URL: "+serviceURL) ;
>
> if ( serviceURL.indexOf('?') >= 0 )
> - throw new QueryExceptionHTTP(-1, "URL already has a query
> string
> ("+serviceURL+")") ;
> + serviceParams = true ;
>
> this.serviceURL = serviceURL ;
> }
> @@ -178,7 +179,7 @@
> if ( count() == 0 )
> target = new URL(serviceURL) ;
> else
> - target = new URL(serviceURL+"?"+qs) ;
> + target = new URL(serviceURL+(serviceParams ? "&" :
> "?")+qs) ;
> }
> catch (MalformedURLException malEx)
> { throw new QueryExceptionHTTP(0, "Malformed URL: "+malEx) ; }
>
>
> This would solve the problems for people wanting to run SPARQL queries
> against SPARQL endpoints which support/require additional query parameters.
>
> The JENA-195 [4] has more details on this and the latest patch attached
> has more changes (the change above is just for HttpQuery.java).
>
> Pending some feedback, I'd like to commit that. I think now it is in a
> much better shape (but if there are better ways, I am happy to further
> improve it).
>
> Let me know what you think.
>
> Thanks,
> Paolo
>
> [1] http://www.w3.org/TR/sparql11-query/#sparqlGrammar
> [2] http://www.w3.org/TR/sparql11-query/#rIRI_REF
> [3]
> https://issues.apache.org/jira/secure/attachment/12510753/JENA-195.patch
> [4] https://issues.apache.org/jira/browse/JENA-195
JENA-195 : a small point of project policy on other-system specific
code.
Posted by Andy Seaborne <an...@apache.org>.
The patch for JENA-195 from Paolo includes a Kasabi-specific example in
the ARQ examples. The example is both specific to Kasabi and to a
particular Kasabi dataset.
There is already a DBpedia example so we have included a
service-specific examples before. DBpedia is used by students and in
student projects so we do get a stream of questions about using it,
especially as it can be peculiar in its timeout behaviour.
However, this case is different because Kasabi requires registration to
obtain an API key and the example does not work without such
registration. The key is required to be in environment variable
KASABI_API_KEY, which is the way all Kasabi client APIs deal with the
API key.
A committer can not test the example unless they register with Kasabi.
This, to me, is the critical difference here.
Personally, I think such examples should go on the Kasabi site and not
in the jena-arq codebase, and eventually Jena examples area which Ian's
been setting up.
If the majority feel this is acceptable, then I will not block it being
included.
Andy
-------- Original Message --------
Subject: Re: Using SERVICE with parameters: HttpException: -1 URL
already has a query string
Date: Thu, 19 Jan 2012 08:40:41 +0000
From: Paolo Castagna <ca...@googlemail.com>
Reply-To: jena-users@incubator.apache.org
To: jena-users@incubator.apache.org
Andy Seaborne wrote:
>> My colleague was using arq.query command line, so he was trying to
>> specify
>> additional query parameters in his SPARQL queries writing:
>>
>> SELECT ... WHERE {
>> SERVICE<...?name1=value1> { ... }
>> SERVICE<...?name2=value2> { ... }
>> }
>>
>> I don't think this is allowed by the SPARQL grammar. Is it?
>
> Have you tried the online query validator?
>
> http://www.sparql.org/query-validator.html
>
Hi Andy,
at first, I did not try the validator (I should have).
I looked at the SPARQL grammar [1] instead.
In particular, the thing I am still not sure is if <http://.../?name=value>
is ok as IRI_REF [2].
The on-line validator has no problems with SPARQL queries containing:
SERVICE <http://.../?name=value>. Good.
ARQ currently throws an exception if someone attempts to use query
parameters in a SERVICE <IRI>.
We now have a patch [3] which, instead of throwing an exception when '?'
is found in the SERVICE <IRI>, keep the query string parameters and send
them to the remote SPARQL endpoint as is. Here is the relevant bit:
Index: src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
===================================================================
--- src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
(revision
1232106)
+++ src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
(working copy)
@@ -68,6 +68,7 @@
String responseMessage = null ;
boolean forcePOST = false ;
String queryString = null ;
+ boolean serviceParams = false ;
//static final String ENC_UTF8 = "UTF-8" ;
@@ -97,7 +98,7 @@
log.trace("URL: "+serviceURL) ;
if ( serviceURL.indexOf('?') >= 0 )
- throw new QueryExceptionHTTP(-1, "URL already has a query
string
("+serviceURL+")") ;
+ serviceParams = true ;
this.serviceURL = serviceURL ;
}
@@ -178,7 +179,7 @@
if ( count() == 0 )
target = new URL(serviceURL) ;
else
- target = new URL(serviceURL+"?"+qs) ;
+ target = new URL(serviceURL+(serviceParams ? "&" :
"?")+qs) ;
}
catch (MalformedURLException malEx)
{ throw new QueryExceptionHTTP(0, "Malformed URL: "+malEx) ; }
This would solve the problems for people wanting to run SPARQL queries
against SPARQL endpoints which support/require additional query parameters.
The JENA-195 [4] has more details on this and the latest patch attached
has more changes (the change above is just for HttpQuery.java).
Pending some feedback, I'd like to commit that. I think now it is in a
much better shape (but if there are better ways, I am happy to further
improve it).
Let me know what you think.
Thanks,
Paolo
[1] http://www.w3.org/TR/sparql11-query/#sparqlGrammar
[2] http://www.w3.org/TR/sparql11-query/#rIRI_REF
[3]
https://issues.apache.org/jira/secure/attachment/12510753/JENA-195.patch
[4] https://issues.apache.org/jira/browse/JENA-195
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Andy Seaborne <an...@apache.org>.
On 19/01/12 08:40, Paolo Castagna wrote:
> Andy Seaborne wrote:
>>> My colleague was using arq.query command line, so he was trying to
>>> specify
>>> additional query parameters in his SPARQL queries writing:
>>>
>>> SELECT ... WHERE {
>>> SERVICE<...?name1=value1> { ... }
>>> SERVICE<...?name2=value2> { ... }
>>> }
>>>
>>> I don't think this is allowed by the SPARQL grammar. Is it?
>>
>> Have you tried the online query validator?
>>
>> http://www.sparql.org/query-validator.html
>>
>
> Hi Andy,
> at first, I did not try the validator (I should have).
> I looked at the SPARQL grammar [1] instead.
>
> In particular, the thing I am still not sure is if<http://.../?name=value>
> is ok as IRI_REF [2].
It is. IRI_REF is any RFC 3986/3987 IRI. Well, it's more general than
that - it's any legal chars. The IRI library is used to parse the IRI
for further checking. <?a=b> is a levgal (relative) IRI.
> The on-line validator has no problems with SPARQL queries containing:
> SERVICE<http://.../?name=value>. Good.
>
> ARQ currently throws an exception if someone attempts to use query
> parameters in a SERVICE<IRI>.
You mentioned this before.
> We now have a patch [3] which, instead of throwing an exception when '?'
> is found in the SERVICE<IRI>, keep the query string parameters and send
> them to the remote SPARQL endpoint as is. Here is the relevant bit:
>
> Index: src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
> ===================================================================
> --- src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java (revision
> 1232106)
> +++ src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java (working copy)
> @@ -68,6 +68,7 @@
> String responseMessage = null ;
> boolean forcePOST = false ;
> String queryString = null ;
> + boolean serviceParams = false ;
>
> //static final String ENC_UTF8 = "UTF-8" ;
>
> @@ -97,7 +98,7 @@
> log.trace("URL: "+serviceURL) ;
>
> if ( serviceURL.indexOf('?')>= 0 )
(old code)
A prize for anyone who a problem with that test ... something quite
obscure ... affects the current code ... :-)
> - throw new QueryExceptionHTTP(-1, "URL already has a query string
> ("+serviceURL+")") ;
> + serviceParams = true ;
>
> this.serviceURL = serviceURL ;
> }
> @@ -178,7 +179,7 @@
> if ( count() == 0 )
> target = new URL(serviceURL) ;
> else
> - target = new URL(serviceURL+"?"+qs) ;
> + target = new URL(serviceURL+(serviceParams ? "&" : "?")+qs) ;
> }
> catch (MalformedURLException malEx)
> { throw new QueryExceptionHTTP(0, "Malformed URL: "+malEx) ; }
>
>
> This would solve the problems for people wanting to run SPARQL queries
> against SPARQL endpoints which support/require additional query parameters.
>
> The JENA-195 [4] has more details on this and the latest patch attached
> has more changes (the change above is just for HttpQuery.java).
>
> Pending some feedback, I'd like to commit that. I think now it is in a
> much better shape (but if there are better ways, I am happy to further
> improve it).
Discussion on jena-dev@i.a.o.
>
> Let me know what you think.
>
> Thanks,
> Paolo
>
> [1] http://www.w3.org/TR/sparql11-query/#sparqlGrammar
> [2] http://www.w3.org/TR/sparql11-query/#rIRI_REF
> [3] https://issues.apache.org/jira/secure/attachment/12510753/JENA-195.patch
> [4] https://issues.apache.org/jira/browse/JENA-195
Andy
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Paolo Castagna <ca...@googlemail.com>.
Andy Seaborne wrote:
>> My colleague was using arq.query command line, so he was trying to
>> specify
>> additional query parameters in his SPARQL queries writing:
>>
>> SELECT ... WHERE {
>> SERVICE<...?name1=value1> { ... }
>> SERVICE<...?name2=value2> { ... }
>> }
>>
>> I don't think this is allowed by the SPARQL grammar. Is it?
>
> Have you tried the online query validator?
>
> http://www.sparql.org/query-validator.html
>
Hi Andy,
at first, I did not try the validator (I should have).
I looked at the SPARQL grammar [1] instead.
In particular, the thing I am still not sure is if <http://.../?name=value>
is ok as IRI_REF [2].
The on-line validator has no problems with SPARQL queries containing:
SERVICE <http://.../?name=value>. Good.
ARQ currently throws an exception if someone attempts to use query
parameters in a SERVICE <IRI>.
We now have a patch [3] which, instead of throwing an exception when '?'
is found in the SERVICE <IRI>, keep the query string parameters and send
them to the remote SPARQL endpoint as is. Here is the relevant bit:
Index: src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java
===================================================================
--- src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java (revision
1232106)
+++ src/main/java/com/hp/hpl/jena/sparql/engine/http/HttpQuery.java (working copy)
@@ -68,6 +68,7 @@
String responseMessage = null ;
boolean forcePOST = false ;
String queryString = null ;
+ boolean serviceParams = false ;
//static final String ENC_UTF8 = "UTF-8" ;
@@ -97,7 +98,7 @@
log.trace("URL: "+serviceURL) ;
if ( serviceURL.indexOf('?') >= 0 )
- throw new QueryExceptionHTTP(-1, "URL already has a query string
("+serviceURL+")") ;
+ serviceParams = true ;
this.serviceURL = serviceURL ;
}
@@ -178,7 +179,7 @@
if ( count() == 0 )
target = new URL(serviceURL) ;
else
- target = new URL(serviceURL+"?"+qs) ;
+ target = new URL(serviceURL+(serviceParams ? "&" : "?")+qs) ;
}
catch (MalformedURLException malEx)
{ throw new QueryExceptionHTTP(0, "Malformed URL: "+malEx) ; }
This would solve the problems for people wanting to run SPARQL queries
against SPARQL endpoints which support/require additional query parameters.
The JENA-195 [4] has more details on this and the latest patch attached
has more changes (the change above is just for HttpQuery.java).
Pending some feedback, I'd like to commit that. I think now it is in a
much better shape (but if there are better ways, I am happy to further
improve it).
Let me know what you think.
Thanks,
Paolo
[1] http://www.w3.org/TR/sparql11-query/#sparqlGrammar
[2] http://www.w3.org/TR/sparql11-query/#rIRI_REF
[3] https://issues.apache.org/jira/secure/attachment/12510753/JENA-195.patch
[4] https://issues.apache.org/jira/browse/JENA-195
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Andy Seaborne <an...@apache.org>.
On 16/01/12 10:12, Paolo Castagna wrote:
> Andy Seaborne wrote:
>> On 16/01/12 08:14, Paolo Castagna wrote:
>>> Thanks Andy.
>>>
>>> Andy Seaborne wrote:
>>>> 1/ Service.java code could look in the Context for additional parameters
>>>> - but how might the code know which endpoints need which parameters?
>>>> (there may be several SERVICE calls to different places in one query -
>>>> don't want APi keys exposed to the wrong places!)
>>>
>>> Indeed, parameters need to be kept and grouped by SERVICE endpoint.
>>> We can use a Map<String, Map<String,List<String>>> to map SERVICE
>>> endpoint URLs to a map of parameter-->values.
>>>
>>> Service.java can then get the Map from the Context and add the
>>> parameters to HttpQuery if there are any.
>>>
>>> If we do just 1/, OpService probably needs to change (to keep the
>>> parameters separate from the SERVICE endpoint URL).
>>
>>
>>>
>>>> 2/ Just not checking for "?" doesn't quite work (need to know there is a
>>>> "?" allowed and swap to"&") but API keys probably shouldn't be in
>>>> SERVICE URIs. Is there any other query string parameters that might
>>>> reasonably and sensibly be in the service URI? If it's just keys, then
>>>> (1) looks better. Or both.
>>>
>>> I can see how this could work and it can be general enough to leave all
>>> the parameters specified by users in their SERVICE<...> call. It needs
>>> to add query=... using either '?' or '&'.
>>>
>>> The specific use case I've seen did not any other query string parameter,
>>> however other SPARQL endpoints might have other additional (and/or
>>> optional) parameters.
>>>
>>>> What might a general design look like?
>>>
>>> We allow users to specify one or more query parameters when they use
>>> SERVICE<...?name=value>.
>>>
>>
>> Don't follow this part.
>>
>> The use of the context is because the query and service IRI (and so the
>> algebra, which is just "the query") then don't need to know about
>> additional, potentially private, parameters.
>>
>> Even if the params/value are passed in the query or URI, the only
>> difference in Params.httpString() as to whether the "first" flag is set
>> or unset initially?
>
> A patch [1] is attached to JENA-195 [2]. This keeps changes to a minimum
> and there is no need to change anything else other than a small addition
> to Service.java (and ARQ.java for a new Symbol).
>
> This solution is available via Java APIs only.
>
> My colleague was using arq.query command line, so he was trying to specify
> additional query parameters in his SPARQL queries writing:
>
> SELECT ... WHERE {
> SERVICE<...?name1=value1> { ... }
> SERVICE<...?name2=value2> { ... }
> }
>
> I don't think this is allowed by the SPARQL grammar. Is it?
Have you tried the online query validator?
http://www.sparql.org/query-validator.html
Andy
>
> Paolo
>
> [1] https://issues.apache.org/jira/secure/attachment/12510689/JENA-195.patch
> [2] https://issues.apache.org/jira/browse/JENA-195
>
>>
>> Andy
>>
>>> We do 1/ keeping all SERVICE endpoint parameters separate from each
>>> others.
>>>
>>>> IIRC someone was looking to rework HttpQuery and see JENA-56.
>>>
>>> Ack.
>>>
>>> Time for a JIRA issue (i.e. improvement).
>>>
>>> Paolo
>>>
>>>>
>>>> Andy
>>>
>>
>
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Paolo Castagna <ca...@googlemail.com>.
Andy Seaborne wrote:
> On 16/01/12 08:14, Paolo Castagna wrote:
>> Thanks Andy.
>>
>> Andy Seaborne wrote:
>>> 1/ Service.java code could look in the Context for additional parameters
>>> - but how might the code know which endpoints need which parameters?
>>> (there may be several SERVICE calls to different places in one query -
>>> don't want APi keys exposed to the wrong places!)
>>
>> Indeed, parameters need to be kept and grouped by SERVICE endpoint.
>> We can use a Map<String, Map<String,List<String>>> to map SERVICE
>> endpoint URLs to a map of parameter-->values.
>>
>> Service.java can then get the Map from the Context and add the
>> parameters to HttpQuery if there are any.
>>
>> If we do just 1/, OpService probably needs to change (to keep the
>> parameters separate from the SERVICE endpoint URL).
>
>
>>
>>> 2/ Just not checking for "?" doesn't quite work (need to know there is a
>>> "?" allowed and swap to"&") but API keys probably shouldn't be in
>>> SERVICE URIs. Is there any other query string parameters that might
>>> reasonably and sensibly be in the service URI? If it's just keys, then
>>> (1) looks better. Or both.
>>
>> I can see how this could work and it can be general enough to leave all
>> the parameters specified by users in their SERVICE<...> call. It needs
>> to add query=... using either '?' or '&'.
>>
>> The specific use case I've seen did not any other query string parameter,
>> however other SPARQL endpoints might have other additional (and/or
>> optional) parameters.
>>
>>> What might a general design look like?
>>
>> We allow users to specify one or more query parameters when they use
>> SERVICE<...?name=value>.
>>
>
> Don't follow this part.
>
> The use of the context is because the query and service IRI (and so the
> algebra, which is just "the query") then don't need to know about
> additional, potentially private, parameters.
>
> Even if the params/value are passed in the query or URI, the only
> difference in Params.httpString() as to whether the "first" flag is set
> or unset initially?
A patch [1] is attached to JENA-195 [2]. This keeps changes to a minimum
and there is no need to change anything else other than a small addition
to Service.java (and ARQ.java for a new Symbol).
This solution is available via Java APIs only.
My colleague was using arq.query command line, so he was trying to specify
additional query parameters in his SPARQL queries writing:
SELECT ... WHERE {
SERVICE <...?name1=value1> { ... }
SERVICE <...?name2=value2> { ... }
}
I don't think this is allowed by the SPARQL grammar. Is it?
Paolo
[1] https://issues.apache.org/jira/secure/attachment/12510689/JENA-195.patch
[2] https://issues.apache.org/jira/browse/JENA-195
>
> Andy
>
>> We do 1/ keeping all SERVICE endpoint parameters separate from each
>> others.
>>
>>> IIRC someone was looking to rework HttpQuery and see JENA-56.
>>
>> Ack.
>>
>> Time for a JIRA issue (i.e. improvement).
>>
>> Paolo
>>
>>>
>>> Andy
>>
>
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Andy Seaborne <an...@apache.org>.
On 16/01/12 08:14, Paolo Castagna wrote:
> Thanks Andy.
>
> Andy Seaborne wrote:
>> 1/ Service.java code could look in the Context for additional parameters
>> - but how might the code know which endpoints need which parameters?
>> (there may be several SERVICE calls to different places in one query -
>> don't want APi keys exposed to the wrong places!)
>
> Indeed, parameters need to be kept and grouped by SERVICE endpoint.
> We can use a Map<String, Map<String,List<String>>> to map SERVICE
> endpoint URLs to a map of parameter-->values.
>
> Service.java can then get the Map from the Context and add the
> parameters to HttpQuery if there are any.
>
> If we do just 1/, OpService probably needs to change (to keep the
> parameters separate from the SERVICE endpoint URL).
>
>> 2/ Just not checking for "?" doesn't quite work (need to know there is a
>> "?" allowed and swap to"&") but API keys probably shouldn't be in
>> SERVICE URIs. Is there any other query string parameters that might
>> reasonably and sensibly be in the service URI? If it's just keys, then
>> (1) looks better. Or both.
>
> I can see how this could work and it can be general enough to leave all
> the parameters specified by users in their SERVICE<...> call. It needs
> to add query=... using either '?' or '&'.
>
> The specific use case I've seen did not any other query string parameter,
> however other SPARQL endpoints might have other additional (and/or
> optional) parameters.
>
>> What might a general design look like?
>
> We allow users to specify one or more query parameters when they use
> SERVICE<...?name=value>.
>
Don't follow this part.
The use of the context is because the query and service IRI (and so the
algebra, which is just "the query") then don't need to know about
additional, potentially private, parameters.
Even if the params/value are passed in the query or URI, the only
difference in Params.httpString() as to whether the "first" flag is set
or unset initially?
Andy
> We do 1/ keeping all SERVICE endpoint parameters separate from each others.
>
>> IIRC someone was looking to rework HttpQuery and see JENA-56.
>
> Ack.
>
> Time for a JIRA issue (i.e. improvement).
>
> Paolo
>
>>
>> Andy
>
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Paolo Castagna <ca...@googlemail.com>.
Paolo Castagna wrote:
> Time for a JIRA issue (i.e. improvement).
https://issues.apache.org/jira/browse/JENA-195
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Paolo Castagna <ca...@googlemail.com>.
Thanks Andy.
Andy Seaborne wrote:
> 1/ Service.java code could look in the Context for additional parameters
> - but how might the code know which endpoints need which parameters?
> (there may be several SERVICE calls to different places in one query -
> don't want APi keys exposed to the wrong places!)
Indeed, parameters need to be kept and grouped by SERVICE endpoint.
We can use a Map<String, Map<String,List<String>>> to map SERVICE
endpoint URLs to a map of parameter-->values.
Service.java can then get the Map from the Context and add the
parameters to HttpQuery if there are any.
If we do just 1/, OpService probably needs to change (to keep the
parameters separate from the SERVICE endpoint URL).
> 2/ Just not checking for "?" doesn't quite work (need to know there is a
> "?" allowed and swap to "&") but API keys probably shouldn't be in
> SERVICE URIs. Is there any other query string parameters that might
> reasonably and sensibly be in the service URI? If it's just keys, then
> (1) looks better. Or both.
I can see how this could work and it can be general enough to leave all
the parameters specified by users in their SERVICE <...> call. It needs
to add query=... using either '?' or '&'.
The specific use case I've seen did not any other query string parameter,
however other SPARQL endpoints might have other additional (and/or
optional) parameters.
> What might a general design look like?
We allow users to specify one or more query parameters when they use
SERVICE <...?name=value>.
We do 1/ keeping all SERVICE endpoint parameters separate from each others.
> IIRC someone was looking to rework HttpQuery and see JENA-56.
Ack.
Time for a JIRA issue (i.e. improvement).
Paolo
>
> Andy
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Andy Seaborne <an...@apache.org>.
On 15/01/12 09:42, Paolo Castagna wrote:
> Andy Seaborne wrote:
>> On 13/01/12 17:19, Paolo Castagna wrote:
>>> Paolo Castagna wrote:
>>>> Or, the SPARQL specs explicitly forbid that... probably here, isn't it?
>>>> http://www.w3.org/TR/sparql11-query/#rIRI_REF
>>>>
>>>> Luckily, QueryEngineHTTP has an addParam(...) method. However, I did
>>>> not find
>>>> a way do go around this within a SPARQL query using SERVICE.
>>>
>>> This isn't as friendly as using SERVICE directly within the SPARQL
>>> queries.
>>>
>>> Would it be possible to change Service.java [1] and allow developers
>>> to set
>>> custom/additional query string parameters on a per service URI basis?
>>> Or, alternatively, HTTP request parameters?
>>>
>>> Paolo
>>>
>>> [1]
>>> http://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/trunk/src/main/java/com/hp/hpl/jena/sparql/engine/http/Service.java
>>>
>>
>> It's a ordering/%-encoding issue. Which version is this? trunk?
>
> Hi Andy,
> I think it was the arq.query command from ARQ included in the Fuseki SNAPSHOT.
>
> But, I had a look at the QueryEngineHTTP and QueryHTTP code in ARQ trunk and
> the same exception will trigger if someone attempts to use SERVICE and a URI
> with some parameters already set (which I think it is forbidden by the spec).
> Isn't it?
>
> Paolo
Of the two different solutions in this thread, it seems that either can
work.
1/ Service.java code could look in the Context for additional parameters
- but how might the code know which endpoints need which parameters?
(there may be several SERVICE calls to different places in one query -
don't want APi keys exposed to the wrong places!)
2/ Just not checking for "?" doesn't quite work (need to know there is a
"?" allowed and swap to "&") but API keys probably shouldn't be in
SERVICE URIs. Is there any other query string parameters that might
reasonably and sensibly be in the service URI? If it's just keys, then
(1) looks better. Or both.
What might a general design look like?
IIRC someone was looking to rework HttpQuery and see JENA-56.
Andy
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Paolo Castagna <ca...@googlemail.com>.
Andy Seaborne wrote:
> On 13/01/12 17:19, Paolo Castagna wrote:
>> Paolo Castagna wrote:
>>> Or, the SPARQL specs explicitly forbid that... probably here, isn't it?
>>> http://www.w3.org/TR/sparql11-query/#rIRI_REF
>>>
>>> Luckily, QueryEngineHTTP has an addParam(...) method. However, I did
>>> not find
>>> a way do go around this within a SPARQL query using SERVICE.
>>
>> This isn't as friendly as using SERVICE directly within the SPARQL
>> queries.
>>
>> Would it be possible to change Service.java [1] and allow developers
>> to set
>> custom/additional query string parameters on a per service URI basis?
>> Or, alternatively, HTTP request parameters?
>>
>> Paolo
>>
>> [1]
>> http://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/trunk/src/main/java/com/hp/hpl/jena/sparql/engine/http/Service.java
>>
>
> It's a ordering/%-encoding issue. Which version is this? trunk?
Hi Andy,
I think it was the arq.query command from ARQ included in the Fuseki SNAPSHOT.
But, I had a look at the QueryEngineHTTP and QueryHTTP code in ARQ trunk and
the same exception will trigger if someone attempts to use SERVICE and a URI
with some parameters already set (which I think it is forbidden by the spec).
Isn't it?
Paolo
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Andy Seaborne <an...@apache.org>.
On 13/01/12 17:19, Paolo Castagna wrote:
> Paolo Castagna wrote:
>> Or, the SPARQL specs explicitly forbid that... probably here, isn't it?
>> http://www.w3.org/TR/sparql11-query/#rIRI_REF
>>
>> Luckily, QueryEngineHTTP has an addParam(...) method. However, I did not find
>> a way do go around this within a SPARQL query using SERVICE.
>
> This isn't as friendly as using SERVICE directly within the SPARQL queries.
>
> Would it be possible to change Service.java [1] and allow developers to set
> custom/additional query string parameters on a per service URI basis?
> Or, alternatively, HTTP request parameters?
>
> Paolo
>
> [1]
> http://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/trunk/src/main/java/com/hp/hpl/jena/sparql/engine/http/Service.java
It's a ordering/%-encoding issue. Which version is this? trunk?
Andy
Re: Using SERVICE with parameters: HttpException: -1 URL already
has a query string
Posted by Paolo Castagna <ca...@googlemail.com>.
Paolo Castagna wrote:
> Or, the SPARQL specs explicitly forbid that... probably here, isn't it?
> http://www.w3.org/TR/sparql11-query/#rIRI_REF
>
> Luckily, QueryEngineHTTP has an addParam(...) method. However, I did not find
> a way do go around this within a SPARQL query using SERVICE.
This isn't as friendly as using SERVICE directly within the SPARQL queries.
Would it be possible to change Service.java [1] and allow developers to set
custom/additional query string parameters on a per service URI basis?
Or, alternatively, HTTP request parameters?
Paolo
[1]
http://svn.apache.org/repos/asf/incubator/jena/Jena2/ARQ/trunk/src/main/java/com/hp/hpl/jena/sparql/engine/http/Service.java