You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Regis Pires Magalhães <re...@gmail.com> on 2012/04/07 23:26:38 UTC
Error in Federated Query
I get a "java.net.NoRouteToHostException: Cannot assign requested address"
exception when I execute que following code:
import com.hp.hpl.jena.query.QueryExecution;
import com.hp.hpl.jena.query.QueryExecutionFactory;
import com.hp.hpl.jena.query.QueryFactory;
import com.hp.hpl.jena.query.QuerySolution;
import com.hp.hpl.jena.query.ResultSet;
import com.hp.hpl.jena.rdf.model.ModelFactory;
import com.hp.hpl.jena.sparql.core.DatasetImpl;
public class Query {
public static void main(String[] args) {
String query = "PREFIX owl: <http://www.w3.org/2002/07/owl#>\n"
+
"PREFIX geopos: <
http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
"select ?s ?lat ?long where {\n" +
" SERVICE <
http://fuseki.mooo.com/lgd-dbpedia/sparql> {\n" +
" select * where {\n" +
" ?s owl:sameAs ?geo .\n" +
" FILTER regex(str(?geo),\"dbpedia\")\n"
+
" } \n" +
" }\n" +
" SERVICE <
http://fuseki.mooo.com/dbpedia-geo/sparql> {\n" +
" select * where{\n" +
" ?geo geopos:lat ?lat ;\n" +
" geopos:long ?long .\n" +
" } \n" +
" } \n" +
"} \n";
long start = System.currentTimeMillis();
QueryExecution exec =
QueryExecutionFactory.create(QueryFactory.create(query),
new
DatasetImpl(ModelFactory.createDefaultModel()));
ResultSet rs = exec.execSelect();
int n = 0;
while (rs.hasNext()) {
rs.next();
System.out.print(++n + " ");
}
long time = System.currentTimeMillis() - start;
System.out.println(n + " results found in " + time + "ms.");
}
}
The same query runs perfectly when a do limit in the first SERVICE select:
String query = "PREFIX owl: <http://www.w3.org/2002/07/owl#>\n"
+
"PREFIX geopos: <
http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
"select ?s ?lat ?long where {\n" +
" SERVICE <
http://fuseki.mooo.com/lgd-dbpedia/sparql> {\n" +
" select * where {\n" +
" ?s owl:sameAs ?geo .\n" +
" FILTER regex(str(?geo),\"dbpedia\")\n"
+
" } *limit 20000* \n" +
" }\n" +
" service <
http://fuseki.mooo.com/dbpedia-geo/sparql> {\n" +
" select * where{\n" +
" ?geo geopos:lat ?lat ;\n" +
" geopos:long ?long .\n" +
" } \n" +
" } \n" +
"} \n";
What can I do to fix the problem? It always occurs when the query is run,
even when the Fuseki Sparql Endpoint is running locally.
I did not observe any problem related to the network. The problem persists
when the code (query) is executed in different hosts.
I am using Jena classes from "fuseki-server.jar" in the client classpath.
I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
The full stack trace is this:
Exception in thread "main" HttpException: java.net.NoRouteToHostException:
Cannot assign requested address: java.net.NoRouteToHostException: Cannot
assign requested address
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:216)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:157)
at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:71)
at
com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:65)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
at Query.main(Query.java:34)
Caused by: java.net.NoRouteToHostException: Cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
at sun.net.www.http.HttpClient.New(HttpClient.java:321)
at sun.net.www.http.HttpClient.New(HttpClient.java:338)
at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:199)
... 14 more
Best regards,
Regis.
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Andy,
I suppose the problem is in the server side (Fuseki) and not in the client
side (ARQ).
I say this because now I have the same data in Fuseki (Jena TDB) and
Virtuoso.
Both are installed in the same host.
I get all the results from Virtuoso, but not from Fuseki using the same
client application and query.
Regards,
Regis.
On Wed, May 2, 2012 at 9:51 AM, Andy Seaborne <an...@apache.org> wrote:
> Hi there,
>
> Sorry for the delay.
>
> I've managed to turn this into a test case.
>
> Using ARQ and fuseki from development (recent snapshot builds will do the
> same), and daft data + query, I can get it reliably to lock up at 50
> triples; 40 does not reliably.
>
> The query is a cross product of the database - the execution plan is going
> to be 2500 calls in the second SERVICE.
>
> I tried:
>
> 1/ putting in a short delay after the execution of service and it works.
> The delay was 5ms - not even an OS timeslice. I guess this causes the OS
> to switch processes or otherwise be able to clear up resources. 1ms
> didn't work. 2ms did -- I only tried a coupel of times though.
>
> 2/ Setting the connection to be closed after use always
> (i.e. HTTP header "Connection: close")
>
> Tentative conclusion: it's the server/OS not being able to sort out
> connections fast enough. ARQ, or the Java runtime, isn't recycling
> connections which maybe because the URL changes everytime but probbaly
> because I don't understand how connection caching is supposed to work in
> Java when naively use java.net.
>
> Not sure of the best solution at the moment.
>
> Andy
>
> ---- Query ----
> PREFIX : <http://example/>
>
> SELECT (count(*) AS ?c)
> FROM <file:D.nt>
> {
> SERVICE <http://localhost:3030/ds/**query<http://localhost:3030/ds/query>>
> { ?s ?p ?o . ?s1 ?p1 ?o1 }
> SERVICE <http://localhost:3030/ds/**query<http://localhost:3030/ds/query>>
> { ?s1 ?p1 ?o1 }
> }
> ---- Query ----
>
> data:
> fuseki --file=D.ttl /ds
>
> ---- Data
> # 50 triples
> @prefix : <http://example/> .
>
> :x :p 1 .
> :x :p 2 .
> :x :p 3 .
> :x :p 4 .
> :x :p 5 .
> :x :p 6 .
> :x :p 7 .
> :x :p 8 .
> :x :p 9 .
> :x :p 10 .
> :x :p 11 .
> :x :p 12 .
> :x :p 13 .
> :x :p 14 .
> :x :p 15 .
> :x :p 16 .
> :x :p 17 .
> :x :p 18 .
> :x :p 19 .
> :x :p 20 .
> :x :p 21 .
> :x :p 22 .
> :x :p 23 .
> :x :p 24 .
> :x :p 25 .
> :x :p 26 .
> :x :p 27 .
> :x :p 28 .
> :x :p 29 .
> :x :p 30 .
>
> :x :p 31 .
> :x :p 32 .
> :x :p 33 .
> :x :p 34 .
> :x :p 35 .
> :x :p 36 .
> :x :p 37 .
> :x :p 38 .
> :x :p 39 .
> :x :p 40 .
>
> :x :p 41 .
> :x :p 42 .
> :x :p 43 .
> :x :p 44 .
> :x :p 45 .
> :x :p 46 .
> :x :p 47 .
> :x :p 48 .
> :x :p 49 .
> :x :p 50 .
>
Re: Error in Federated Query
Posted by Andy Seaborne <an...@apache.org>.
Hi there,
Sorry for the delay.
I've managed to turn this into a test case.
Using ARQ and fuseki from development (recent snapshot builds will do
the same), and daft data + query, I can get it reliably to lock up at 50
triples; 40 does not reliably.
The query is a cross product of the database - the execution plan is
going to be 2500 calls in the second SERVICE.
I tried:
1/ putting in a short delay after the execution of service and it works.
The delay was 5ms - not even an OS timeslice. I guess this causes the
OS to switch processes or otherwise be able to clear up resources. 1ms
didn't work. 2ms did -- I only tried a coupel of times though.
2/ Setting the connection to be closed after use always
(i.e. HTTP header "Connection: close")
Tentative conclusion: it's the server/OS not being able to sort out
connections fast enough. ARQ, or the Java runtime, isn't recycling
connections which maybe because the URL changes everytime but probbaly
because I don't understand how connection caching is supposed to work in
Java when naively use java.net.
Not sure of the best solution at the moment.
Andy
---- Query ----
PREFIX : <http://example/>
SELECT (count(*) AS ?c)
FROM <file:D.nt>
{
SERVICE <http://localhost:3030/ds/query> { ?s ?p ?o . ?s1 ?p1 ?o1 }
SERVICE <http://localhost:3030/ds/query> { ?s1 ?p1 ?o1 }
}
---- Query ----
data:
fuseki --file=D.ttl /ds
---- Data
# 50 triples
@prefix : <http://example/> .
:x :p 1 .
:x :p 2 .
:x :p 3 .
:x :p 4 .
:x :p 5 .
:x :p 6 .
:x :p 7 .
:x :p 8 .
:x :p 9 .
:x :p 10 .
:x :p 11 .
:x :p 12 .
:x :p 13 .
:x :p 14 .
:x :p 15 .
:x :p 16 .
:x :p 17 .
:x :p 18 .
:x :p 19 .
:x :p 20 .
:x :p 21 .
:x :p 22 .
:x :p 23 .
:x :p 24 .
:x :p 25 .
:x :p 26 .
:x :p 27 .
:x :p 28 .
:x :p 29 .
:x :p 30 .
:x :p 31 .
:x :p 32 .
:x :p 33 .
:x :p 34 .
:x :p 35 .
:x :p 36 .
:x :p 37 .
:x :p 38 .
:x :p 39 .
:x :p 40 .
:x :p 41 .
:x :p 42 .
:x :p 43 .
:x :p 44 .
:x :p 45 .
:x :p 46 .
:x :p 47 .
:x :p 48 .
:x :p 49 .
:x :p 50 .
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Some more information:
When I stop the client (via Ctrl + C) the server unlocks and logs the last
client request:
20:03:07 INFO Fuseki :: [253] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Fs+%3Fp+%3Fo+%7D%0ALIMIT+++10%0A
20:03:07 INFO Fuseki :: [254] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FSteubenville%252C_Ohio%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FSteubenville%252C_Ohio%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
After that the server answers other requests done when it was locked and it
is also ready to process new requests.
Regards,
Regis.
2012/4/14 Regis Pires Magalhães <re...@gmail.com>
> Andy,
> Some answers to your questions...
>
>
>
>> and there should be a lot more results?
>
>
> Yes, it should return 43016 results.
>
> Did you try limiting the SERVICE calls a bit by restricting the number of
>> results with a LIMIT on a subquery inside a SERVICE, particualrly the first
>> one.
>
>
> Yes, if I do a limit, it works!
>
> (long term - switching to a hash join for some or all of these queries
>> would be better for some cases, but that's long term)
>
>
> Yes, it is true.
>
>
> If that's the end of the log, it looks as if all requests have been
>> responded to. (are there any "[X] GET" with no corresponding "[X] OK"?)
>
>
> Yes, that's the end of the log and there were 252 GETs and 252 OKs.
>
>
> The server is limited to the number of threads it has (5 by default but
>> there's no point giving it huge numbers as it'll run out of system
>> resources).
>
>
> Ok.
>
>
> Does nginx have any rate control configured? The execution strategy ARQ
>> is using is to issue a large number of hopefully quite grounded queries.
>
>
> I'm not using Nginx until the problem is solved. The queries are sent
> directly to Fuseki now (http://fuseki.mooo.com:8080/).
>
>
>
> (client side query explain might show the corresponding actions:
>> http://incubator.apache.org/**jena/documentation/query/**explain.html<http://incubator.apache.org/jena/documentation/query/explain.html>
>> )
>
>
>
> Comparing end of log (server) and end of explain (client):
>
> end of server log:
>
> ...
> 19:04:03 INFO Fuseki :: [251] OK/select
> 19:04:03 INFO Fuseki :: [251] 200 OK
> 19:04:03 INFO Fuseki :: [252] GET
> http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
> 19:04:03 INFO Fuseki :: [252] Query = SELECT * WHERE { <
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
> 19:04:03 INFO Fuseki :: [252] OK/select
> 19:04:03 INFO Fuseki :: [252] 200 OK
>
>
> End of client explain query using ARQ:
>
> ...
> 19:04:07 INFO exec :: HTTP
> SELECT *
> WHERE
> { <
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
> <
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long
> }
> *19:04:07 INFO exec :: HTTP*
> * SELECT **
> * WHERE*
> * { <http://dbpedia.org/resource/Steubenville%2C_Ohio> <
> http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .*
> * <http://dbpedia.org/resource/Steubenville%2C_Ohio> <
> http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long*
> * }*
>
> The client sends the "last" query, but the server was already locked and
> does not log that query (the red one).
> The server is really locked because it does not process any additional
> requests.
>
>
> Regards,
> Regis.
>
>
>
>
> On Sat, Apr 14, 2012 at 2:16 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 13/04/12 16:29, Regis Pires Magalhăes wrote:
>>
>>> *"RC = 500 : Direct buffer memory"* only occured when Fuseki started with
>>>
>>> "exec java -Xmx1200M -jar ...".
>>>
>>> When I use *"exec java -Xmx2400M -jar ..."* even without *
>>> -XX:MaxDirectMemorySize* or *jetty_config.xml*, the client shows 18
>>> results
>>>
>>> and stops (no exception occurs). The server gives no error.
>>>
>>> Client output:
>>> java -cp jena-fuseki-0.2.1-incubating-**server.jar:. Query2
>>> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
>>>
>>
>> and there should be a lot more results?
>>
>> Did you try limiting the SERVICE calls a bit by restricting the number of
>> results with a LIMIT on a subquery inside a SERVICE, particualrly the first
>> one.
>>
>> (long term - switching to a hash join for some or all of these queries
>> would be better for some cases, but that's long term)
>>
>>
>> Server log:
>>> ...
>>> 12:24:48 INFO Fuseki :: [251] OK/select
>>> 12:24:48 INFO Fuseki :: [251] 200 OK
>>> 12:24:48 INFO Fuseki :: [252] GET
>>> http://fuseki.mooo.com:8080/**dbpedia-geo/sparql?query=**
>>> SELECT++*%0AWHERE%0A++%7B+%**3Chttp%3A%2F%2Fdbpedia.org%**
>>> 2Fresource%2FNorth_Versailles_**Township%252C_Allegheny_**
>>> County%252C_Pennsylvania%3E+%**3Chttp%3A%2F%2Fwww.w3.org%**
>>> 2F2003%2F01%2Fgeo%2Fwgs84_pos%**23lat%3E+%3Flat+.%0A++++%**
>>> 3Chttp%3A%2F%2Fdbpedia.org%**2Fresource%2FNorth_Versailles_**
>>> Township%252C_Allegheny_**County%252C_Pennsylvania%3E+%**
>>> 3Chttp%3A%2F%2Fwww.w3.org%**2F2003%2F01%2Fgeo%2Fwgs84_pos%**
>>> 23long%3E+%3Flong%0A++%7D%0A<http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A>
>>> 12:24:48 INFO Fuseki :: [252] Query = SELECT * WHERE {<
>>> http://dbpedia.org/resource/**North_Versailles_Township%2C_**
>>> Allegheny_County%2C_**Pennsylvania<http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
>>> >
>>> <http://www.w3.org/2003/01/**geo/wgs84_pos#lat<http://www.w3.org/2003/01/geo/wgs84_pos#lat>>
>>> ?lat .<
>>> http://dbpedia.org/resource/**North_Versailles_Township%2C_**
>>> Allegheny_County%2C_**Pennsylvania<http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
>>> >
>>> <http://www.w3.org/2003/01/**geo/wgs84_pos#long<http://www.w3.org/2003/01/geo/wgs84_pos#long>>
>>> ?long }
>>> 12:24:48 INFO Fuseki :: [252] OK/select
>>> 12:24:48 INFO Fuseki :: [252] 200 OK
>>>
>>
>> If that's the end of the log, it looks as if all requests have been
>> responded to. (are there any "[X] GET" with no corresponding "[X] OK"?)
>>
>> The server is limited to the number of threads it has (5 by default but
>> there's no point giving it huge numbers as it'll run out of system
>> resources).
>>
>> Does nginx have any rate control configured? The execution strategy ARQ
>> is using is to issue a large number of hopefully quite grounded queries.
>>
>> (client side query explain might show the corresponding actions:
>> http://incubator.apache.org/**jena/documentation/query/**explain.html<http://incubator.apache.org/jena/documentation/query/explain.html>
>> )
>>
>> Andy
>>
>>
>>>
>>>
>>> Regards,
>>> Regis.
>>>
>>
>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Andy,
Some answers to your questions...
> and there should be a lot more results?
Yes, it should return 43016 results.
Did you try limiting the SERVICE calls a bit by restricting the number of
> results with a LIMIT on a subquery inside a SERVICE, particualrly the first
> one.
Yes, if I do a limit, it works!
(long term - switching to a hash join for some or all of these queries
> would be better for some cases, but that's long term)
Yes, it is true.
If that's the end of the log, it looks as if all requests have been
> responded to. (are there any "[X] GET" with no corresponding "[X] OK"?)
Yes, that's the end of the log and there were 252 GETs and 252 OKs.
The server is limited to the number of threads it has (5 by default but
> there's no point giving it huge numbers as it'll run out of system
> resources).
Ok.
Does nginx have any rate control configured? The execution strategy ARQ is
> using is to issue a large number of hopefully quite grounded queries.
I'm not using Nginx until the problem is solved. The queries are sent
directly to Fuseki now (http://fuseki.mooo.com:8080/).
(client side query explain might show the corresponding actions:
> http://incubator.apache.org/**jena/documentation/query/**explain.html<http://incubator.apache.org/jena/documentation/query/explain.html>
> )
Comparing end of log (server) and end of explain (client):
end of server log:
...
19:04:03 INFO Fuseki :: [251] OK/select
19:04:03 INFO Fuseki :: [251] 200 OK
19:04:03 INFO Fuseki :: [252] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
19:04:03 INFO Fuseki :: [252] Query = SELECT * WHERE { <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
19:04:03 INFO Fuseki :: [252] OK/select
19:04:03 INFO Fuseki :: [252] 200 OK
End of client explain query using ARQ:
...
19:04:07 INFO exec :: HTTP
SELECT *
WHERE
{ <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
<
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long
}
*19:04:07 INFO exec :: HTTP*
* SELECT **
* WHERE*
* { <http://dbpedia.org/resource/Steubenville%2C_Ohio> <
http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .*
* <http://dbpedia.org/resource/Steubenville%2C_Ohio> <
http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long*
* }*
The client sends the "last" query, but the server was already locked and
does not log that query (the red one).
The server is really locked because it does not process any additional
requests.
Regards,
Regis.
On Sat, Apr 14, 2012 at 2:16 PM, Andy Seaborne <an...@apache.org> wrote:
> On 13/04/12 16:29, Regis Pires Magalhăes wrote:
>
>> *"RC = 500 : Direct buffer memory"* only occured when Fuseki started with
>>
>> "exec java -Xmx1200M -jar ...".
>>
>> When I use *"exec java -Xmx2400M -jar ..."* even without *
>> -XX:MaxDirectMemorySize* or *jetty_config.xml*, the client shows 18
>> results
>>
>> and stops (no exception occurs). The server gives no error.
>>
>> Client output:
>> java -cp jena-fuseki-0.2.1-incubating-**server.jar:. Query2
>> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
>>
>
> and there should be a lot more results?
>
> Did you try limiting the SERVICE calls a bit by restricting the number of
> results with a LIMIT on a subquery inside a SERVICE, particualrly the first
> one.
>
> (long term - switching to a hash join for some or all of these queries
> would be better for some cases, but that's long term)
>
>
> Server log:
>> ...
>> 12:24:48 INFO Fuseki :: [251] OK/select
>> 12:24:48 INFO Fuseki :: [251] 200 OK
>> 12:24:48 INFO Fuseki :: [252] GET
>> http://fuseki.mooo.com:8080/**dbpedia-geo/sparql?query=**
>> SELECT++*%0AWHERE%0A++%7B+%**3Chttp%3A%2F%2Fdbpedia.org%**
>> 2Fresource%2FNorth_Versailles_**Township%252C_Allegheny_**
>> County%252C_Pennsylvania%3E+%**3Chttp%3A%2F%2Fwww.w3.org%**
>> 2F2003%2F01%2Fgeo%2Fwgs84_pos%**23lat%3E+%3Flat+.%0A++++%**
>> 3Chttp%3A%2F%2Fdbpedia.org%**2Fresource%2FNorth_Versailles_**
>> Township%252C_Allegheny_**County%252C_Pennsylvania%3E+%**
>> 3Chttp%3A%2F%2Fwww.w3.org%**2F2003%2F01%2Fgeo%2Fwgs84_pos%**
>> 23long%3E+%3Flong%0A++%7D%0A<http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A>
>> 12:24:48 INFO Fuseki :: [252] Query = SELECT * WHERE {<
>> http://dbpedia.org/resource/**North_Versailles_Township%2C_**
>> Allegheny_County%2C_**Pennsylvania<http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
>> >
>> <http://www.w3.org/2003/01/**geo/wgs84_pos#lat<http://www.w3.org/2003/01/geo/wgs84_pos#lat>>
>> ?lat .<
>> http://dbpedia.org/resource/**North_Versailles_Township%2C_**
>> Allegheny_County%2C_**Pennsylvania<http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
>> >
>> <http://www.w3.org/2003/01/**geo/wgs84_pos#long<http://www.w3.org/2003/01/geo/wgs84_pos#long>>
>> ?long }
>> 12:24:48 INFO Fuseki :: [252] OK/select
>> 12:24:48 INFO Fuseki :: [252] 200 OK
>>
>
> If that's the end of the log, it looks as if all requests have been
> responded to. (are there any "[X] GET" with no corresponding "[X] OK"?)
>
> The server is limited to the number of threads it has (5 by default but
> there's no point giving it huge numbers as it'll run out of system
> resources).
>
> Does nginx have any rate control configured? The execution strategy ARQ
> is using is to issue a large number of hopefully quite grounded queries.
>
> (client side query explain might show the corresponding actions:
> http://incubator.apache.org/**jena/documentation/query/**explain.html<http://incubator.apache.org/jena/documentation/query/explain.html>
> )
>
> Andy
>
>
>>
>>
>> Regards,
>> Regis.
>>
>
Re: Error in Federated Query
Posted by Andy Seaborne <an...@apache.org>.
On 13/04/12 16:29, Regis Pires Magalhães wrote:
> *"RC = 500 : Direct buffer memory"* only occured when Fuseki started with
> "exec java -Xmx1200M -jar ...".
>
> When I use *"exec java -Xmx2400M -jar ..."* even without *
> -XX:MaxDirectMemorySize* or *jetty_config.xml*, the client shows 18 results
> and stops (no exception occurs). The server gives no error.
>
> Client output:
> java -cp jena-fuseki-0.2.1-incubating-server.jar:. Query2
> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
and there should be a lot more results?
Did you try limiting the SERVICE calls a bit by restricting the number
of results with a LIMIT on a subquery inside a SERVICE, particualrly the
first one.
(long term - switching to a hash join for some or all of these queries
would be better for some cases, but that's long term)
> Server log:
> ...
> 12:24:48 INFO Fuseki :: [251] OK/select
> 12:24:48 INFO Fuseki :: [251] 200 OK
> 12:24:48 INFO Fuseki :: [252] GET
> http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
> 12:24:48 INFO Fuseki :: [252] Query = SELECT * WHERE {<
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .<
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
> 12:24:48 INFO Fuseki :: [252] OK/select
> 12:24:48 INFO Fuseki :: [252] 200 OK
If that's the end of the log, it looks as if all requests have been
responded to. (are there any "[X] GET" with no corresponding "[X] OK"?)
The server is limited to the number of threads it has (5 by default but
there's no point giving it huge numbers as it'll run out of system
resources).
Does nginx have any rate control configured? The execution strategy ARQ
is using is to issue a large number of hopefully quite grounded queries.
(client side query explain might show the corresponding actions:
http://incubator.apache.org/jena/documentation/query/explain.html)
Andy
>
>
>
> Regards,
> Regis.
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
*"RC = 500 : Direct buffer memory"* only occured when Fuseki started with
"exec java -Xmx1200M -jar ...".
When I use *"exec java -Xmx2400M -jar ..."* even without *
-XX:MaxDirectMemorySize* or *jetty_config.xml*, the client shows 18 results
and stops (no exception occurs). The server gives no error.
Client output:
java -cp jena-fuseki-0.2.1-incubating-server.jar:. Query2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Server log:
...
12:24:48 INFO Fuseki :: [251] OK/select
12:24:48 INFO Fuseki :: [251] 200 OK
12:24:48 INFO Fuseki :: [252] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
12:24:48 INFO Fuseki :: [252] Query = SELECT * WHERE { <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
12:24:48 INFO Fuseki :: [252] OK/select
12:24:48 INFO Fuseki :: [252] 200 OK
Regards,
Regis.
2012/4/13 Regis Pires Magalhães <re...@gmail.com>
> Andy,
> Thank you again!
>
> In my tests:
>
> 1 - Starting Fuseki with "exec java *-XX:MaxDirectMemorySize=2147483648-Xmx4800M
> * -jar ..." shows only 18 results again and the client stops and doesn't
> show any result.
>
> Server log:
> ...
> 11:55:59 INFO Fuseki :: [251] Query = SELECT * WHERE { <
> http://dbpedia.org/resource/Penn_Hills_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
> http://dbpedia.org/resource/Penn_Hills_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
> 11:55:59 INFO Fuseki :: [251] OK/select
> 11:55:59 INFO Fuseki :: [251] 200 OK
> 11:55:59 INFO Fuseki :: [252] GET
> http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
> 11:55:59 INFO Fuseki :: [252] Query = SELECT * WHERE { <
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
> http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
> <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
> 11:55:59 INFO Fuseki :: [252] OK/select
> 11:55:59 INFO Fuseki :: [252] 200 OK
>
>
> 2 - Using *S**electChannelConnector* with the suggested configuration: it
> seems that the server answers the first request, but the client stops and
> doesn't show any result.
>
> Server log:
> ...
> 11:22:41 INFO Server :: Jetty server config file =
> jetty-config.xml
> 11:22:42 INFO Server :: Dataset path = /dblp
> 11:22:42 INFO Server :: Dataset path = /dailymed
> 11:22:42 INFO Server :: Dataset path = /diseasome
> 11:22:42 INFO Server :: Dataset path = /drugbank
> 11:22:42 INFO Server :: Dataset path = /sider
> 11:22:42 INFO Server :: Dataset path = /researchers
> 11:22:42 INFO Server :: Dataset path = /books
> 11:22:42 INFO Server :: Dataset path = /lgd-dbpedia
> 11:22:42 INFO Server :: Dataset path = /dbpedia-geo
> 11:22:42 INFO Server :: Fuseki 0.2.1-incubating-SNAPSHOT
> 11:22:42 INFO Server :: Jetty 7.x.y-SNAPSHOT
> 11:22:42 INFO Server :: Started 2012/04/13 11:22:42 BRT on
> port 8080
> 11:22:49 INFO Fuseki :: [1] GET
> http://fuseki.mooo.com:8080/lgd-dbpedia/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Fs+%3Chttp%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs%3E+%3Fgeo%0A++++FILTER+regex%28str%28%3Fgeo%29%2C+%22dbpedia%22%29%0A++%7D%0A
> 11:22:49 INFO Fuseki :: [1] Query = SELECT * WHERE { ?s <
> http://www.w3.org/2002/07/owl#sameAs> ?geo FILTER regex(str(?geo),
> "dbpedia") }
> 11:22:49 INFO Fuseki :: [1] OK/select
>
>
> *jetty_config.xml:*
> <?xml version="1.0"?>
> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "
> http://www.eclipse.org/jetty/configure.dtd">
>
> <!--
> Reference: http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax
> http://wiki.eclipse.org/Jetty/Reference/jetty.xml
> -->
>
> <Configure id="Fuseki" class="org.eclipse.jetty.server.Server">
> <Call name="addConnector">
> <Arg>
> <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
> <Set name="port">8080</Set>
> <Set name="maxIdleTime">0</Set>
> <!-- All connectors -->
> <Set name="requestHeaderSize">65536</Set> <!-- 64*1024 -->
> <Set name="requestBufferSize">5242880</Set> <!-- 5*1024*1024 -->
> <Set name="responseBufferSize">5242880</Set> <!-- 5*1024*1024 -->
> </New>
> </Arg>
> </Call>
> </Configure>
>
> 3 - Using *BlockingChannelConnector* with the suggested configuration:
>
> a. First when I tested with -Xmx1200M there was an OutOfMemory
> exception...
>
> 11:43:05 INFO Fuseki :: [166] Query = SELECT * WHERE { <
> http://dbpedia.org/resource/Takoma_Park%2C_Maryland> <
> http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
> http://dbpedia.org/resource/Takoma_Park%2C_Maryland> <
> http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
> 11:43:05 INFO Fuseki :: [166] OK/select
> 11:43:06 INFO Fuseki :: [166] 200 OK
> 11:43:06 INFO Fuseki :: [167] GET
> http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FRockville%252C_Maryland%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FRockville%252C_Maryland%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
> 11:43:06 INFO Fuseki :: [167] Query = SELECT * WHERE { <
> http://dbpedia.org/resource/Rockville%2C_Maryland> <
> http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
> http://dbpedia.org/resource/Rockville%2C_Maryland> <
> http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
> 11:43:06 INFO Fuseki :: [167] OK/select
> 11:43:06 WARN Fuseki :: [167] RC = 500 : Java heap space
> Exception in thread "qtp547896362-12" java.lang.OutOfMemoryError: GC
> overhead limit exceeded
> Exception in thread "qtp547896362-125" java.lang.OutOfMemoryError: GC
> overhead limit exceeded
> Exception in thread "qtp547896362-23" java.lang.OutOfMemoryError: GC
> overhead limit exceeded
> Exception in thread "qtp547896362-64" java.lang.OutOfMemoryError: GC
> overhead limit exceeded
>
>
> b. When I use "exec java -Xmx2400M -jar ..."
> The server gets 20 results. After that the client stops.
>
>
> *jetty_config.xml:*
> <?xml version="1.0"?>
> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "
> http://www.eclipse.org/jetty/configure.dtd">
>
> <!--
> Reference: http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax
> http://wiki.eclipse.org/Jetty/Reference/jetty.xml
> -->
>
> <Configure id="Fuseki" class="org.eclipse.jetty.server.Server">
> <Call name="addConnector">
> <Arg>
> <New class="org.eclipse.jetty.server.nio.BlockingChannelConnector">
> * <Set name="useDirectBuffers">false</Set>*
> <Set name="port">8080</Set>
>
> <!-- All connectors -->
> <Set name="requestHeaderSize">65536</Set> <!-- 64*1024 -->
> <Set name="requestBufferSize">5242880</Set> <!-- 5*1024*1024 -->
> <Set name="responseBufferSize">5242880</Set> <!-- 5*1024*1024 -->
> </New>
> </Arg>
> </Call>
> </Configure>
>
>
>
> Regards,
> Regis.
>
>
>
> On Fri, Apr 13, 2012 at 6:24 AM, Paolo Castagna <
> castagna.lists@googlemail.com> wrote:
>
>> Andy Seaborne wrote:
>> > When we have encountered this here, we set that 1G (default is 64K or
>> > something small) and had no problems. (we=Epimoprhics, here=a
>> > production server for a customer).
>>
>> ...
>>
>> > "BlockingChannelConnector" does by default and is used by Fuseki by
>> > default. <Set name="useDirectBuffer">false</Set> turns use of direct
>> > memory off.
>> >
>> > Any differences in performance are not measurable except under high
>> load.
>>
>> ...
>>
>> This is useful to know, thanks for sharing Andy (and for the complete
>> jetty-config.xml example). Useful.
>>
>> Paolo
>>
>
>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Andy,
Thank you again!
In my tests:
1 - Starting Fuseki with "exec java *-XX:MaxDirectMemorySize=2147483648
-Xmx4800M* -jar ..." shows only 18 results again and the client stops and
doesn't show any result.
Server log:
...
11:55:59 INFO Fuseki :: [251] Query = SELECT * WHERE { <
http://dbpedia.org/resource/Penn_Hills_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/Penn_Hills_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
11:55:59 INFO Fuseki :: [251] OK/select
11:55:59 INFO Fuseki :: [251] 200 OK
11:55:59 INFO Fuseki :: [252] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FNorth_Versailles_Township%252C_Allegheny_County%252C_Pennsylvania%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
11:55:59 INFO Fuseki :: [252] Query = SELECT * WHERE { <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/North_Versailles_Township%2C_Allegheny_County%2C_Pennsylvania>
<http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
11:55:59 INFO Fuseki :: [252] OK/select
11:55:59 INFO Fuseki :: [252] 200 OK
2 - Using *S**electChannelConnector* with the suggested configuration: it
seems that the server answers the first request, but the client stops and
doesn't show any result.
Server log:
...
11:22:41 INFO Server :: Jetty server config file =
jetty-config.xml
11:22:42 INFO Server :: Dataset path = /dblp
11:22:42 INFO Server :: Dataset path = /dailymed
11:22:42 INFO Server :: Dataset path = /diseasome
11:22:42 INFO Server :: Dataset path = /drugbank
11:22:42 INFO Server :: Dataset path = /sider
11:22:42 INFO Server :: Dataset path = /researchers
11:22:42 INFO Server :: Dataset path = /books
11:22:42 INFO Server :: Dataset path = /lgd-dbpedia
11:22:42 INFO Server :: Dataset path = /dbpedia-geo
11:22:42 INFO Server :: Fuseki 0.2.1-incubating-SNAPSHOT
11:22:42 INFO Server :: Jetty 7.x.y-SNAPSHOT
11:22:42 INFO Server :: Started 2012/04/13 11:22:42 BRT on
port 8080
11:22:49 INFO Fuseki :: [1] GET
http://fuseki.mooo.com:8080/lgd-dbpedia/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Fs+%3Chttp%3A%2F%2Fwww.w3.org%2F2002%2F07%2Fowl%23sameAs%3E+%3Fgeo%0A++++FILTER+regex%28str%28%3Fgeo%29%2C+%22dbpedia%22%29%0A++%7D%0A
11:22:49 INFO Fuseki :: [1] Query = SELECT * WHERE { ?s <
http://www.w3.org/2002/07/owl#sameAs> ?geo FILTER regex(str(?geo),
"dbpedia") }
11:22:49 INFO Fuseki :: [1] OK/select
*jetty_config.xml:*
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "
http://www.eclipse.org/jetty/configure.dtd">
<!--
Reference: http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax
http://wiki.eclipse.org/Jetty/Reference/jetty.xml
-->
<Configure id="Fuseki" class="org.eclipse.jetty.server.Server">
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
<Set name="port">8080</Set>
<Set name="maxIdleTime">0</Set>
<!-- All connectors -->
<Set name="requestHeaderSize">65536</Set> <!-- 64*1024 -->
<Set name="requestBufferSize">5242880</Set> <!-- 5*1024*1024 -->
<Set name="responseBufferSize">5242880</Set> <!-- 5*1024*1024 -->
</New>
</Arg>
</Call>
</Configure>
3 - Using *BlockingChannelConnector* with the suggested configuration:
a. First when I tested with -Xmx1200M there was an OutOfMemory exception...
11:43:05 INFO Fuseki :: [166] Query = SELECT * WHERE { <
http://dbpedia.org/resource/Takoma_Park%2C_Maryland> <
http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/Takoma_Park%2C_Maryland> <
http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
11:43:05 INFO Fuseki :: [166] OK/select
11:43:06 INFO Fuseki :: [166] 200 OK
11:43:06 INFO Fuseki :: [167] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FRockville%252C_Maryland%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FRockville%252C_Maryland%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
11:43:06 INFO Fuseki :: [167] Query = SELECT * WHERE { <
http://dbpedia.org/resource/Rockville%2C_Maryland> <
http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/Rockville%2C_Maryland> <
http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long }
11:43:06 INFO Fuseki :: [167] OK/select
11:43:06 WARN Fuseki :: [167] RC = 500 : Java heap space
Exception in thread "qtp547896362-12" java.lang.OutOfMemoryError: GC
overhead limit exceeded
Exception in thread "qtp547896362-125" java.lang.OutOfMemoryError: GC
overhead limit exceeded
Exception in thread "qtp547896362-23" java.lang.OutOfMemoryError: GC
overhead limit exceeded
Exception in thread "qtp547896362-64" java.lang.OutOfMemoryError: GC
overhead limit exceeded
b. When I use "exec java -Xmx2400M -jar ..."
The server gets 20 results. After that the client stops.
*jetty_config.xml:*
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "
http://www.eclipse.org/jetty/configure.dtd">
<!--
Reference: http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax
http://wiki.eclipse.org/Jetty/Reference/jetty.xml
-->
<Configure id="Fuseki" class="org.eclipse.jetty.server.Server">
<Call name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.nio.BlockingChannelConnector">
* <Set name="useDirectBuffers">false</Set>*
<Set name="port">8080</Set>
<!-- All connectors -->
<Set name="requestHeaderSize">65536</Set> <!-- 64*1024 -->
<Set name="requestBufferSize">5242880</Set> <!-- 5*1024*1024 -->
<Set name="responseBufferSize">5242880</Set> <!-- 5*1024*1024 -->
</New>
</Arg>
</Call>
</Configure>
Regards,
Regis.
On Fri, Apr 13, 2012 at 6:24 AM, Paolo Castagna <
castagna.lists@googlemail.com> wrote:
> Andy Seaborne wrote:
> > When we have encountered this here, we set that 1G (default is 64K or
> > something small) and had no problems. (we=Epimoprhics, here=a
> > production server for a customer).
>
> ...
>
> > "BlockingChannelConnector" does by default and is used by Fuseki by
> > default. <Set name="useDirectBuffer">false</Set> turns use of direct
> > memory off.
> >
> > Any differences in performance are not measurable except under high load.
>
> ...
>
> This is useful to know, thanks for sharing Andy (and for the complete
> jetty-config.xml example). Useful.
>
> Paolo
>
Re: Error in Federated Query
Posted by Paolo Castagna <ca...@googlemail.com>.
Andy Seaborne wrote:
> When we have encountered this here, we set that 1G (default is 64K or
> something small) and had no problems. (we=Epimoprhics, here=a
> production server for a customer).
...
> "BlockingChannelConnector" does by default and is used by Fuseki by
> default. <Set name="useDirectBuffer">false</Set> turns use of direct
> memory off.
>
> Any differences in performance are not measurable except under high load.
...
This is useful to know, thanks for sharing Andy (and for the complete
jetty-config.xml example). Useful.
Paolo
Re: Error in Federated Query
Posted by Andy Seaborne <an...@apache.org>.
Regis,
When we have encountered this here, we set that 1G (default is 64K or
something small) and had no problems. (we=Epimoprhics, here=a
production server for a customer).
There is also "plan B" which is don't use that bit of Jetty.
The fuseki server command takes an additional argument of
--jetty-config=jetty-config.xml
which allows a different Jetty configuration and in particular, either
switch off use of direct memory or switch to a different connector
implementation altogether.
"SelectChannelConnector" does not use direct memory at all.
"BlockingChannelConnector" does by default and is used by Fuseki by
default. <Set name="useDirectBuffer">false</Set> turns use of direct
memory off.
Any differences in performance are not measurable except under high load.
Extract:
<Arg>
<New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
<Set name="port">3030</Set>
<Set name="maxIdleTime">0</Set>
<!-- All connectors -->
<Set name="requestHeaderSize">65536</Set>
<Set name="requestBufferSize">5242880</Set>
<Set name="responseBufferSize">5242880</Set>
</New>
...
Full example:
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN"
"http://www.eclipse.org/jetty/configure.dtd">
<!--
Reference: http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax
http://wiki.eclipse.org/Jetty/Reference/jetty.xml
-->
<Configure id="Fuseki" class="org.eclipse.jetty.server.Server">
<Call name="addConnector">
<Arg>
<!-- org.eclipse.jetty.server.nio.BlockingChannelConnector -->
<!-- org.eclipse.jetty.server.nio.SelectChannelConnector -->
<New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
<!-- BlockingChannelConnector specific:
<Set name="useDirectBuffer">false</Set>
-->
<Set name="port">3535</Set>
<Set name="maxIdleTime">0</Set>
<!-- All connectors -->
<Set name="requestHeaderSize">65536</Set> <!-- 64*1024 -->
<Set name="requestBufferSize">5242880</Set> <!-- 5*1024*1024 -->
<Set name="responseBufferSize">5242880</Set> <!-- 5*1024*1024 -->
</New>
</Arg>
</Call>
</Configure>
Andy
On 12/04/12 22:00, Regis Pires Magalhães wrote:
> Andy and Rob,
> Thank you for your help.
> I tried -XX:MaxDirectMemorySize to a higher value, but I have not been
> successful.
>
> After I ran the same query many times using Virtuoso as the Sparql Endpoint
> with the same data and there was no error. Virtuoso and Fuseki are
> installed on the same host (Intel Core i7 - 16 GB RAM).
> So I think the problem is on the server side and not in the client app that
> uses Jena ARQ.
>
> Is there anything else that can be configured in Fuseki in order to get all
> the results?
> I would really like to use Fuseki.
>
> Regards,
> Regis.
>
>
> On Mon, Apr 9, 2012 at 4:07 PM, Andy Seaborne<an...@apache.org> wrote:
>
>> On 09/04/12 19:47, Rob Vesse wrote:
>>
>>> Hi Regis
>>>
>>> Please see this thread where I experienced a similar issue:
>>>
>>> https://issues.apache.org/**jira/browse/JENA-181<https://issues.apache.org/jira/browse/JENA-181>
>>>
>>> A couple of possible solutions is either to insert delays between
>>> queries (not possible in your scenario) or to set
>>> -XX:MaxDirectMemorySize to a higher value than the default
>>>
>>> Andy - is it possible that the Service code is not calling close() on
>>> the query iterators in a timely fashion (i.e. keeping too many
>>> connections open thus exhausting the direct memory buffer) as that
>>> turned out to be the main culprit in my case from re-reading that thread.
>>>
>>
>> It's possible but not in the simplest way because QueryIterService calls
>> QueryIter.materialize so it should be reading the results when the results
>> are first received. Looking at the code, exhausting the iterator should
>> call the close operation. I tried breakpointing a single SERVICE call and
>> it did hit the close breakpoint.
>>
>> But in the server, direct memory does not always seem to be returned or
>> retuned fast enough. We have had to -XX:MaxDirectMemorySize (plan B: use
>> a separate jetty config and pick a different connector implementation).
>>
>> Having two SERVICE calls means the second is called for each match from
>> the first.
>>
>> An experiment is to put a LIMIT on the first call and see what numbers
>> matter.
>>
>> Andy
>>
>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Andy and Rob,
Thank you for your help.
I tried -XX:MaxDirectMemorySize to a higher value, but I have not been
successful.
After I ran the same query many times using Virtuoso as the Sparql Endpoint
with the same data and there was no error. Virtuoso and Fuseki are
installed on the same host (Intel Core i7 - 16 GB RAM).
So I think the problem is on the server side and not in the client app that
uses Jena ARQ.
Is there anything else that can be configured in Fuseki in order to get all
the results?
I would really like to use Fuseki.
Regards,
Regis.
On Mon, Apr 9, 2012 at 4:07 PM, Andy Seaborne <an...@apache.org> wrote:
> On 09/04/12 19:47, Rob Vesse wrote:
>
>> Hi Regis
>>
>> Please see this thread where I experienced a similar issue:
>>
>> https://issues.apache.org/**jira/browse/JENA-181<https://issues.apache.org/jira/browse/JENA-181>
>>
>> A couple of possible solutions is either to insert delays between
>> queries (not possible in your scenario) or to set
>> -XX:MaxDirectMemorySize to a higher value than the default
>>
>> Andy - is it possible that the Service code is not calling close() on
>> the query iterators in a timely fashion (i.e. keeping too many
>> connections open thus exhausting the direct memory buffer) as that
>> turned out to be the main culprit in my case from re-reading that thread.
>>
>
> It's possible but not in the simplest way because QueryIterService calls
> QueryIter.materialize so it should be reading the results when the results
> are first received. Looking at the code, exhausting the iterator should
> call the close operation. I tried breakpointing a single SERVICE call and
> it did hit the close breakpoint.
>
> But in the server, direct memory does not always seem to be returned or
> retuned fast enough. We have had to -XX:MaxDirectMemorySize (plan B: use
> a separate jetty config and pick a different connector implementation).
>
> Having two SERVICE calls means the second is called for each match from
> the first.
>
> An experiment is to put a LIMIT on the first call and see what numbers
> matter.
>
> Andy
>
Re: Error in Federated Query
Posted by Andy Seaborne <an...@apache.org>.
On 09/04/12 19:47, Rob Vesse wrote:
> Hi Regis
>
> Please see this thread where I experienced a similar issue:
>
> https://issues.apache.org/jira/browse/JENA-181
>
> A couple of possible solutions is either to insert delays between
> queries (not possible in your scenario) or to set
> -XX:MaxDirectMemorySize to a higher value than the default
>
> Andy - is it possible that the Service code is not calling close() on
> the query iterators in a timely fashion (i.e. keeping too many
> connections open thus exhausting the direct memory buffer) as that
> turned out to be the main culprit in my case from re-reading that thread.
It's possible but not in the simplest way because QueryIterService calls
QueryIter.materialize so it should be reading the results when the
results are first received. Looking at the code, exhausting the
iterator should call the close operation. I tried breakpointing a
single SERVICE call and it did hit the close breakpoint.
But in the server, direct memory does not always seem to be returned or
retuned fast enough. We have had to -XX:MaxDirectMemorySize (plan B:
use a separate jetty config and pick a different connector implementation).
Having two SERVICE calls means the second is called for each match from
the first.
An experiment is to put a LIMIT on the first call and see what numbers
matter.
Andy
Re: Error in Federated Query
Posted by Rob Vesse <ra...@ecs.soton.ac.uk>.
Hi Regis
Please see this thread where I experienced a similar issue:
https://issues.apache.org/jira/browse/JENA-181
A couple of possible solutions is either to insert delays between
queries (not possible in your scenario) or to set
-XX:MaxDirectMemorySize to a higher value than the default
Andy - is it possible that the Service code is not calling close() on
the query iterators in a timely fashion (i.e. keeping too many
connections open thus exhausting the direct memory buffer) as that
turned out to be the main culprit in my case from re-reading that thread.
Rob
On 4/9/12 11:36 AM, Regis Pires Magalhães wrote:
> Andy,
> Yes, I'm running the server and using the client with direct access to
> Fuseki (i.e. without proxy).
>
> When the Exception occurs in client side, the server outputs to log:
>
> *...*
> *14:23:22 INFO Fuseki :: [443] 200 OK*
> *14:23:22 INFO Fuseki :: [444] GET
> http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonaldsonville%252C_Louisiana%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonaldsonville%252C_Louisiana%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
> *
> *14:23:22 INFO Fuseki :: [444] Query = SELECT * WHERE {<
> http://dbpedia.org/resource/Donaldsonville%2C_Louisiana> <
> http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .<
> http://dbpedia.org/resource/Donaldsonville%2C_Louisiana> <
> http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long } *
> *14:23:22 INFO Fuseki :: [444] OK/select*
> *14:23:22 WARN Fuseki :: [444] RC = 500 : Direct buffer
> memory*
>
> When I use SERVICE SILENT, the client outputs:
> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 15:34:19 WARN QueryIterService
> :: SERVICE: HttpException: 500 Server Error
> 15:34:20 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:21 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:21 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:22 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:23 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:23 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:24 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> 15:34:25 WARN QueryIterService :: SERVICE: HttpException: 500 Server
> Error
> ...
>
> How can I avoid "Direct buffer memory" problem?
> The client uses *jena-arq-2.9.0-incubating.jar* and the server uses *
> Fuseki-0.2.1-incubating-SNAPSHOT*.
>
> Regards,
> Regis.
>
>
> On Mon, Apr 9, 2012 at 1:17 PM, Andy Seaborne<an...@apache.org> wrote:
>
>> Regis,
>>
>> What is fuseki.mooo.com? Are you running that server?
>>
>> 500 means the server is unhappy -- the server log file may give further
>> details.
>>
>> I don't know why you are seeing large numbers of connections - the client
>> code uses vanilla java.net HTTP code so it should reuse connections or
>> close them (at least in the ARQ 2.9.0).
>>
>> I'm not familiar with NginX as a proxy. ARQ does try to execute more
>> specific SERVICE requests that use one vague one - sometimes the right
>> thing to do, sometimes not. The engine can't tell.
>>
>> We did recently (post 2.9.0 release - available in 2.9.1-SNAPSHOT) change
>> the SERVICE handling code. If you say "SERVICE SILENT" it will read all
>> the results from a call before moving on - that might help.
>>
>> Andy
>>
>>
>> On 08/04/12 14:00, Regis Pires Magalhães wrote:
>>
>>> This code can be used to see the HttpException (500 Server Error) remotely
>>> (only 18 results found before the exception occurs):
>>>
>>> import com.hp.hpl.jena.query.**QueryExecution;
>>> import com.hp.hpl.jena.query.**QueryExecutionFactory;
>>> import com.hp.hpl.jena.query.**QueryFactory;
>>> import com.hp.hpl.jena.query.**QuerySolution;
>>> import com.hp.hpl.jena.query.**ResultSet;
>>> import com.hp.hpl.jena.rdf.model.**ModelFactory;
>>> import com.hp.hpl.jena.sparql.core.**DatasetImpl;
>>>
>>> public class Query {
>>> public static void main(String[] args) {
>>> String query = "PREFIX owl:<http://www.w3.org/2002/**
>>> 07/owl#<http://www.w3.org/2002/07/owl#>>\n"
>>> +
>>> "PREFIX geopos:<
>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>> +
>>> "select ?s ?lat ?long where {\n" +
>>> " SERVICE
>>> <http://*fuseki.mooo.com:8080***/lgd-dbpedia/sparql>
>>>
>>> {\n" +
>>> " select * where {\n" +
>>> " ?s owl:sameAs ?geo .\n" +
>>> " FILTER regex(str(?geo),\"dbpedia\")\
>>> **n"
>>> +
>>> " } \n" +
>>> " }\n" +
>>> " SERVICE
>>> <http://*fuseki.mooo.com:8080***/dbpedia-geo/sparql>
>>>
>>> {\n" +
>>> " select * where{\n" +
>>> " ?geo geopos:lat ?lat ;\n" +
>>> " geopos:long ?long .\n" +
>>> " } \n" +
>>> " } \n" +
>>> "} \n";
>>>
>>> long start = System.currentTimeMillis();
>>> QueryExecution exec =
>>> QueryExecutionFactory.create(**QueryFactory.create(query),
>>> new
>>> DatasetImpl(ModelFactory.**createDefaultModel()));
>>> ResultSet rs = exec.execSelect();
>>> int n = 0;
>>> while (rs.hasNext()) {
>>> rs.next();
>>> System.out.print(++n + " ");
>>> }
>>> long time = System.currentTimeMillis() - start;
>>> System.out.println(n + " results found in " + time +
>>> "ms.");
>>> }
>>> }
>>>
>>>
>>> Abraços,
>>> Regis.
>>>
>>>
>>> 2012/4/8 Regis Pires Magalhães<re...@gmail.com>
>>> Andy,
>>>> The exception is always the same NoRouteToHostException and we the only
>>>> proxy we use between the client and the Fuseki server is a reverse proxy
>>>> (NginX).
>>>>
>>>> I've also observed that the number of network connections with the server
>>>> increases from 0 to a bit more than 28000, when the exception
>>>> occurs. Almost all of the connections are TIME_WAIT.
>>>>
>>>> When I try a direct connection to Fuseki (even locally using 127.0.0.1 as
>>>> the address to the Sparql Endpoint) I only get 17 results (the number of
>>>> connections increases from 0 to 212) and the exception changes to:
>>>>
>>>> Exception in thread "main" HttpException: HttpException: 500 Server Error
>>>> : HttpException: 500 Server Error
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execCommon(**
>>>> HttpQuery.java:375)
>>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>>> HttpQuery.java:202)
>>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.exec(HttpQuery.**
>>>> java:157)
>>>> at com.hp.hpl.jena.sparql.engine.**http.Service.exec(Service.**java:71)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**main.iterator.**
>>>> QueryIterService.nextStage(**QueryIterService.java:53)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>>> makeNextStage(**QueryIterRepeatApply.java:113)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>>> hasNextBinding(**QueryIterRepeatApply.java:65)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterConvert.**
>>>> hasNextBinding(**QueryIterConvert.java:65)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**ResultSetStream.hasNext(**
>>>> ResultSetStream.java:72)
>>>> at Query.main(Query.java:42)
>>>> Caused by: HttpException: 500 Server Error
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execCommon(**
>>>> HttpQuery.java:320)
>>>> ... 15 more
>>>>
>>>> How could I get all the results?
>>>>
>>>> Regards,
>>>> Regis.
>>>>
>>>>
>>>>
>>>> On Sun, Apr 8, 2012 at 7:38 AM, Andy Seaborne<an...@apache.org> wrote:
>>>>
>>>> On 08/04/12 01:30, Regis Pires Magalhăes wrote:
>>>>> I've got a sucessful result when executing *outside* our intranet
>>>>>> (LAN).
>>>>>>
>>>>>> All the 43016 results were returned.
>>>>>> It seems that there are so many requests (when executing in the Local
>>>>>> Network) that the server or the client can not properly
>>>>>> handle them. Suggestions? I could not receive more than 5000 results
>>>>>> when
>>>>>> querying inside the LAN. Results come much faster, but the exception
>>>>>> always
>>>>>> occurs.
>>>>>>
>>>>>> Regis.
>>>>>>
>>>>>>
>>>>> Is it the same exception? NoRouteToHostException isn't usually a sign
>>>>> of
>>>>> overload - it's a sign that the JVM inside the firewall can't contact
>>>>> the
>>>>> machine outside. Your proxy may be doing funny stuff though.
>>>>>
>>>>> Do you have proxy settings given to the JVM?it should work.
>>>>>
>>>>> -Dhttp.proxyHost=...
>>>>> -Dhttp.proxyPort=...
>>>>>
>>>>> or
>>>>> -DsocksProxyHost=...
>>>>> -DsocksProxyPort=...
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>>
>>>>>> 2012/4/7 Regis Pires Magalhăes<regispiresmag@gmail.****com<
>>>>>> regispiresmag@gmail.com>
>>>>>>
>>>>>>>
>>>>>> I get a "java.net.****NoRouteToHostException: Cannot assign requested
>>>>>>> address"
>>>>>>> exception when I execute que following code:
>>>>>>>
>>>>>>> import com.hp.hpl.jena.query.****QueryExecution;
>>>>>>> import com.hp.hpl.jena.query.****QueryExecutionFactory;
>>>>>>> import com.hp.hpl.jena.query.****QueryFactory;
>>>>>>> import com.hp.hpl.jena.query.****QuerySolution;
>>>>>>> import com.hp.hpl.jena.query.****ResultSet;
>>>>>>> import com.hp.hpl.jena.rdf.model.****ModelFactory;
>>>>>>> import com.hp.hpl.jena.sparql.core.****DatasetImpl;
>>>>>>>
>>>>>>>
>>>>>>> public class Query {
>>>>>>> public static void main(String[] args) {
>>>>>>> String query = "PREFIX owl:<
>>>>>>> http://www.w3.org/2002/07/owl#****<http://www.w3.org/2002/07/owl#**>>\n"
>>>>>>> +
>>>>>>> "PREFIX geopos:<
>>>>>>> http://www.w3.org/2003/01/geo/****wgs84_pos#<http://www.w3.org/2003/01/geo/**wgs84_pos#>
>>>>>>> <http://www.w3.**org/2003/01/geo/wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>
>>>>>>>>> \**n"
>>>>>>> +
>>>>>>> "select ?s ?lat ?long where {\n" +
>>>>>>> " SERVICE<
>>>>>>> http://fuseki.mooo.com/lgd-****dbpedia/sparql<http://fuseki.mooo.com/lgd-**dbpedia/sparql>
>>>>>>> <http://fuseki.**mooo.com/lgd-dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>
>>>>>>> {\n" +
>>>>>>> " select * where {\n" +
>>>>>>> " ?s owl:sameAs ?geo .\n" +
>>>>>>> " FILTER
>>>>>>> regex(str(?geo),\"dbpedia\")\****n" +
>>>>>>>
>>>>>>> " } \n" +
>>>>>>> " }\n" +
>>>>>>> " SERVICE<
>>>>>>> http://fuseki.mooo.com/****dbpedia-geo/sparql<http://fuseki.mooo.com/**dbpedia-geo/sparql>
>>>>>>> <http://**fuseki.mooo.com/dbpedia-geo/**sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>
>>>>>>> {\n" +
>>>>>>> " select * where{\n" +
>>>>>>> " ?geo geopos:lat ?lat ;\n" +
>>>>>>> " geopos:long ?long .\n" +
>>>>>>> " } \n" +
>>>>>>> " } \n" +
>>>>>>> "} \n";
>>>>>>>
>>>>>>> long start = System.currentTimeMillis();
>>>>>>> QueryExecution exec =
>>>>>>> QueryExecutionFactory.create(****QueryFactory.create(query),
>>>>>>> new
>>>>>>> DatasetImpl(ModelFactory.****createDefaultModel()));
>>>>>>>
>>>>>>> ResultSet rs = exec.execSelect();
>>>>>>> int n = 0;
>>>>>>> while (rs.hasNext()) {
>>>>>>> rs.next();
>>>>>>> System.out.print(++n + " ");
>>>>>>> }
>>>>>>> long time = System.currentTimeMillis() - start;
>>>>>>> System.out.println(n + " results found in " + time +
>>>>>>> "ms.");
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> The same query runs perfectly when a do limit in the first SERVICE
>>>>>>> select:
>>>>>>>
>>>>>>> String query = "PREFIX owl:<
>>>>>>> http://www.w3.org/2002/07/owl#****<http://www.w3.org/2002/07/owl#**>>\n"
>>>>>>> +
>>>>>>> "PREFIX geopos:<
>>>>>>> http://www.w3.org/2003/01/geo/****wgs84_pos#<http://www.w3.org/2003/01/geo/**wgs84_pos#>
>>>>>>> <http://www.w3.**org/2003/01/geo/wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>
>>>>>>>>> \**n"
>>>>>>> +
>>>>>>> "select ?s ?lat ?long where {\n" +
>>>>>>> " SERVICE<
>>>>>>> http://fuseki.mooo.com/lgd-****dbpedia/sparql<http://fuseki.mooo.com/lgd-**dbpedia/sparql>
>>>>>>> <http://fuseki.**mooo.com/lgd-dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>
>>>>>>> {\n" +
>>>>>>> " select * where {\n" +
>>>>>>> " ?s owl:sameAs ?geo .\n" +
>>>>>>> " FILTER
>>>>>>> regex(str(?geo),\"dbpedia\")\****n" +
>>>>>>>
>>>>>>> " } *limit 20000* \n" +
>>>>>>>
>>>>>>> " }\n" +
>>>>>>> " service<
>>>>>>> http://fuseki.mooo.com/****dbpedia-geo/sparql<http://fuseki.mooo.com/**dbpedia-geo/sparql>
>>>>>>> <http://**fuseki.mooo.com/dbpedia-geo/**sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>
>>>>>>> {\n" +
>>>>>>> " select * where{\n" +
>>>>>>> " ?geo geopos:lat ?lat ;\n" +
>>>>>>> " geopos:long ?long .\n" +
>>>>>>> " } \n" +
>>>>>>> " } \n" +
>>>>>>> "} \n";
>>>>>>>
>>>>>>>
>>>>>>> What can I do to fix the problem? It always occurs when the query is
>>>>>>> run,
>>>>>>> even when the Fuseki Sparql Endpoint is running locally.
>>>>>>> I did not observe any problem related to the network. The problem
>>>>>>> persists
>>>>>>> when the code (query) is executed in different hosts.
>>>>>>> I am using Jena classes from "fuseki-server.jar" in the client
>>>>>>> classpath.
>>>>>>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
>>>>>>> The full stack trace is this:
>>>>>>>
>>>>>>> Exception in thread "main" HttpException:
>>>>>>> java.net.****NoRouteToHostException: Cannot assign requested address:
>>>>>>> java.net.****NoRouteToHostException: Cannot assign requested address
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****http.HttpQuery.execGet(**
>>>>>>> HttpQuery.java:216)
>>>>>>> at com.hp.hpl.jena.sparql.engine.****http.HttpQuery.exec(**
>>>>>>> HttpQuery.**
>>>>>>> java:157)
>>>>>>> at com.hp.hpl.jena.sparql.engine.****http.Service.exec(Service.**
>>>>>>> java:71)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****main.iterator.**
>>>>>>> QueryIterService.nextStage(****QueryIterService.java:53)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIterRepeatApply.**
>>>>>>> makeNextStage(****QueryIterRepeatApply.java:113)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIterRepeatApply.**
>>>>>>> hasNextBinding(****QueryIterRepeatApply.java:65)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIterConvert.**
>>>>>>> hasNextBinding(****QueryIterConvert.java:65)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIteratorWrapper.**
>>>>>>> hasNextBinding(****QueryIteratorWrapper.java:40)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIteratorWrapper.**
>>>>>>> hasNextBinding(****QueryIteratorWrapper.java:40)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****ResultSetStream.hasNext(**
>>>>>>>
>>>>>>> ResultSetStream.java:72)
>>>>>>> at Query.main(Query.java:34)
>>>>>>> Caused by: java.net.****NoRouteToHostException: Cannot assign
>>>>>>> requested
>>>>>>> address
>>>>>>> at java.net.PlainSocketImpl.****socketConnect(Native Method)
>>>>>>> at
>>>>>>> java.net.****AbstractPlainSocketImpl.****doConnect(**
>>>>>>> AbstractPlainSocketImpl.java:****327)
>>>>>>> at
>>>>>>> java.net.****AbstractPlainSocketImpl.****connectToAddress(**
>>>>>>> AbstractPlainSocketImpl.java:****193)
>>>>>>> at
>>>>>>> java.net.****AbstractPlainSocketImpl.****connect(**
>>>>>>> AbstractPlainSocketImpl.java:****180)
>>>>>>> at java.net.SocksSocketImpl.****connect(SocksSocketImpl.java:****384)
>>>>>>> at java.net.Socket.connect(****Socket.java:546)
>>>>>>> at java.net.Socket.connect(****Socket.java:495)
>>>>>>> at sun.net.NetworkClient.****doConnect(NetworkClient.java:****178)
>>>>>>> at sun.net.www.http.HttpClient.****openServer(HttpClient.java:****
>>>>>>> 409)
>>>>>>> at sun.net.www.http.HttpClient.****openServer(HttpClient.java:****
>>>>>>> 530)
>>>>>>> at sun.net.www.http.HttpClient.<****init>(HttpClient.java:240)
>>>>>>> at sun.net.www.http.HttpClient.****New(HttpClient.java:321)
>>>>>>> at sun.net.www.http.HttpClient.****New(HttpClient.java:338)
>>>>>>> at
>>>>>>> sun.net.www.protocol.http.****HttpURLConnection.****
>>>>>>> getNewHttpClient(**
>>>>>>> HttpURLConnection.java:935)
>>>>>>> at
>>>>>>> sun.net.www.protocol.http.****HttpURLConnection.****plainConnect(**
>>>>>>> HttpURLConnection.java:876)
>>>>>>> at
>>>>>>> sun.net.www.protocol.http.****HttpURLConnection.connect(**
>>>>>>> HttpURLConnection.java:801)
>>>>>>> at
>>>>>>> com.hp.hpl.jena.sparql.engine.****http.HttpQuery.execGet(**
>>>>>>>
>>>>>>> HttpQuery.java:199)
>>>>>>> ... 14 more
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Regis.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Andy,
Yes, I'm running the server and using the client with direct access to
Fuseki (i.e. without proxy).
When the Exception occurs in client side, the server outputs to log:
*...*
*14:23:22 INFO Fuseki :: [443] 200 OK*
*14:23:22 INFO Fuseki :: [444] GET
http://fuseki.mooo.com:8080/dbpedia-geo/sparql?query=SELECT++*%0AWHERE%0A++%7B+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonaldsonville%252C_Louisiana%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23lat%3E+%3Flat+.%0A++++%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDonaldsonville%252C_Louisiana%3E+%3Chttp%3A%2F%2Fwww.w3.org%2F2003%2F01%2Fgeo%2Fwgs84_pos%23long%3E+%3Flong%0A++%7D%0A
*
*14:23:22 INFO Fuseki :: [444] Query = SELECT * WHERE { <
http://dbpedia.org/resource/Donaldsonville%2C_Louisiana> <
http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat . <
http://dbpedia.org/resource/Donaldsonville%2C_Louisiana> <
http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long } *
*14:23:22 INFO Fuseki :: [444] OK/select*
*14:23:22 WARN Fuseki :: [444] RC = 500 : Direct buffer
memory*
When I use SERVICE SILENT, the client outputs:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 15:34:19 WARN QueryIterService
:: SERVICE: HttpException: 500 Server Error
15:34:20 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:21 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:21 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:22 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:23 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:23 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:24 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
15:34:25 WARN QueryIterService :: SERVICE: HttpException: 500 Server
Error
...
How can I avoid "Direct buffer memory" problem?
The client uses *jena-arq-2.9.0-incubating.jar* and the server uses *
Fuseki-0.2.1-incubating-SNAPSHOT*.
Regards,
Regis.
On Mon, Apr 9, 2012 at 1:17 PM, Andy Seaborne <an...@apache.org> wrote:
> Regis,
>
> What is fuseki.mooo.com? Are you running that server?
>
> 500 means the server is unhappy -- the server log file may give further
> details.
>
> I don't know why you are seeing large numbers of connections - the client
> code uses vanilla java.net HTTP code so it should reuse connections or
> close them (at least in the ARQ 2.9.0).
>
> I'm not familiar with NginX as a proxy. ARQ does try to execute more
> specific SERVICE requests that use one vague one - sometimes the right
> thing to do, sometimes not. The engine can't tell.
>
> We did recently (post 2.9.0 release - available in 2.9.1-SNAPSHOT) change
> the SERVICE handling code. If you say "SERVICE SILENT" it will read all
> the results from a call before moving on - that might help.
>
> Andy
>
>
> On 08/04/12 14:00, Regis Pires Magalhães wrote:
>
>> This code can be used to see the HttpException (500 Server Error) remotely
>> (only 18 results found before the exception occurs):
>>
>> import com.hp.hpl.jena.query.**QueryExecution;
>> import com.hp.hpl.jena.query.**QueryExecutionFactory;
>> import com.hp.hpl.jena.query.**QueryFactory;
>> import com.hp.hpl.jena.query.**QuerySolution;
>> import com.hp.hpl.jena.query.**ResultSet;
>> import com.hp.hpl.jena.rdf.model.**ModelFactory;
>> import com.hp.hpl.jena.sparql.core.**DatasetImpl;
>>
>> public class Query {
>> public static void main(String[] args) {
>> String query = "PREFIX owl:<http://www.w3.org/2002/**
>> 07/owl# <http://www.w3.org/2002/07/owl#>>\n"
>> +
>> "PREFIX geopos:<
>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>> +
>> "select ?s ?lat ?long where {\n" +
>> " SERVICE
>> <http://*fuseki.mooo.com:8080***/lgd-dbpedia/sparql>
>>
>> {\n" +
>> " select * where {\n" +
>> " ?s owl:sameAs ?geo .\n" +
>> " FILTER regex(str(?geo),\"dbpedia\")\
>> **n"
>> +
>> " } \n" +
>> " }\n" +
>> " SERVICE
>> <http://*fuseki.mooo.com:8080***/dbpedia-geo/sparql>
>>
>> {\n" +
>> " select * where{\n" +
>> " ?geo geopos:lat ?lat ;\n" +
>> " geopos:long ?long .\n" +
>> " } \n" +
>> " } \n" +
>> "} \n";
>>
>> long start = System.currentTimeMillis();
>> QueryExecution exec =
>> QueryExecutionFactory.create(**QueryFactory.create(query),
>> new
>> DatasetImpl(ModelFactory.**createDefaultModel()));
>> ResultSet rs = exec.execSelect();
>> int n = 0;
>> while (rs.hasNext()) {
>> rs.next();
>> System.out.print(++n + " ");
>> }
>> long time = System.currentTimeMillis() - start;
>> System.out.println(n + " results found in " + time +
>> "ms.");
>> }
>> }
>>
>>
>> Abraços,
>> Regis.
>>
>>
>> 2012/4/8 Regis Pires Magalhães<re...@gmail.com>
>> >
>>
>> Andy,
>>> The exception is always the same NoRouteToHostException and we the only
>>> proxy we use between the client and the Fuseki server is a reverse proxy
>>> (NginX).
>>>
>>> I've also observed that the number of network connections with the server
>>> increases from 0 to a bit more than 28000, when the exception
>>> occurs. Almost all of the connections are TIME_WAIT.
>>>
>>> When I try a direct connection to Fuseki (even locally using 127.0.0.1 as
>>> the address to the Sparql Endpoint) I only get 17 results (the number of
>>> connections increases from 0 to 212) and the exception changes to:
>>>
>>> Exception in thread "main" HttpException: HttpException: 500 Server Error
>>> : HttpException: 500 Server Error
>>> at
>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execCommon(**
>>> HttpQuery.java:375)
>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>> HttpQuery.java:202)
>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.exec(HttpQuery.**
>>> java:157)
>>> at com.hp.hpl.jena.sparql.engine.**http.Service.exec(Service.**java:71)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**main.iterator.**
>>> QueryIterService.nextStage(**QueryIterService.java:53)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>> makeNextStage(**QueryIterRepeatApply.java:113)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>> hasNextBinding(**QueryIterRepeatApply.java:65)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterConvert.**
>>> hasNextBinding(**QueryIterConvert.java:65)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**ResultSetStream.hasNext(**
>>> ResultSetStream.java:72)
>>> at Query.main(Query.java:42)
>>> Caused by: HttpException: 500 Server Error
>>> at
>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execCommon(**
>>> HttpQuery.java:320)
>>> ... 15 more
>>>
>>> How could I get all the results?
>>>
>>> Regards,
>>> Regis.
>>>
>>>
>>>
>>> On Sun, Apr 8, 2012 at 7:38 AM, Andy Seaborne<an...@apache.org> wrote:
>>>
>>> On 08/04/12 01:30, Regis Pires Magalhăes wrote:
>>>>
>>>> I've got a sucessful result when executing *outside* our intranet
>>>>> (LAN).
>>>>>
>>>>> All the 43016 results were returned.
>>>>> It seems that there are so many requests (when executing in the Local
>>>>> Network) that the server or the client can not properly
>>>>> handle them. Suggestions? I could not receive more than 5000 results
>>>>> when
>>>>> querying inside the LAN. Results come much faster, but the exception
>>>>> always
>>>>> occurs.
>>>>>
>>>>> Regis.
>>>>>
>>>>>
>>>> Is it the same exception? NoRouteToHostException isn't usually a sign
>>>> of
>>>> overload - it's a sign that the JVM inside the firewall can't contact
>>>> the
>>>> machine outside. Your proxy may be doing funny stuff though.
>>>>
>>>> Do you have proxy settings given to the JVM?it should work.
>>>>
>>>> -Dhttp.proxyHost=...
>>>> -Dhttp.proxyPort=...
>>>>
>>>> or
>>>> -DsocksProxyHost=...
>>>> -DsocksProxyPort=...
>>>>
>>>> Andy
>>>>
>>>>
>>>>
>>>>> 2012/4/7 Regis Pires Magalhăes<regispiresmag@gmail.****com<
>>>>> regispiresmag@gmail.com>
>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I get a "java.net.****NoRouteToHostException: Cannot assign requested
>>>>>>
>>>>>> address"
>>>>>> exception when I execute que following code:
>>>>>>
>>>>>> import com.hp.hpl.jena.query.****QueryExecution;
>>>>>> import com.hp.hpl.jena.query.****QueryExecutionFactory;
>>>>>> import com.hp.hpl.jena.query.****QueryFactory;
>>>>>> import com.hp.hpl.jena.query.****QuerySolution;
>>>>>> import com.hp.hpl.jena.query.****ResultSet;
>>>>>> import com.hp.hpl.jena.rdf.model.****ModelFactory;
>>>>>> import com.hp.hpl.jena.sparql.core.****DatasetImpl;
>>>>>>
>>>>>>
>>>>>> public class Query {
>>>>>> public static void main(String[] args) {
>>>>>> String query = "PREFIX owl:<
>>>>>> http://www.w3.org/2002/07/owl#**** <http://www.w3.org/2002/07/owl#**>>\n"
>>>>>> +
>>>>>> "PREFIX geopos:<
>>>>>> http://www.w3.org/2003/01/geo/****wgs84_pos#<http://www.w3.org/2003/01/geo/**wgs84_pos#>
>>>>>> <http://www.w3.**org/2003/01/geo/wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>
>>>>>> >>\**n"
>>>>>>
>>>>>> +
>>>>>> "select ?s ?lat ?long where {\n" +
>>>>>> " SERVICE<
>>>>>> http://fuseki.mooo.com/lgd-****dbpedia/sparql<http://fuseki.mooo.com/lgd-**dbpedia/sparql>
>>>>>> <http://fuseki.**mooo.com/lgd-dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>
>>>>>> >>
>>>>>>
>>>>>> {\n" +
>>>>>> " select * where {\n" +
>>>>>> " ?s owl:sameAs ?geo .\n" +
>>>>>> " FILTER
>>>>>> regex(str(?geo),\"dbpedia\")\****n" +
>>>>>>
>>>>>> " } \n" +
>>>>>> " }\n" +
>>>>>> " SERVICE<
>>>>>> http://fuseki.mooo.com/****dbpedia-geo/sparql<http://fuseki.mooo.com/**dbpedia-geo/sparql>
>>>>>> <http://**fuseki.mooo.com/dbpedia-geo/**sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>
>>>>>> >>
>>>>>>
>>>>>> {\n" +
>>>>>> " select * where{\n" +
>>>>>> " ?geo geopos:lat ?lat ;\n" +
>>>>>> " geopos:long ?long .\n" +
>>>>>> " } \n" +
>>>>>> " } \n" +
>>>>>> "} \n";
>>>>>>
>>>>>> long start = System.currentTimeMillis();
>>>>>> QueryExecution exec =
>>>>>> QueryExecutionFactory.create(****QueryFactory.create(query),
>>>>>> new
>>>>>> DatasetImpl(ModelFactory.****createDefaultModel()));
>>>>>>
>>>>>> ResultSet rs = exec.execSelect();
>>>>>> int n = 0;
>>>>>> while (rs.hasNext()) {
>>>>>> rs.next();
>>>>>> System.out.print(++n + " ");
>>>>>> }
>>>>>> long time = System.currentTimeMillis() - start;
>>>>>> System.out.println(n + " results found in " + time +
>>>>>> "ms.");
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> The same query runs perfectly when a do limit in the first SERVICE
>>>>>> select:
>>>>>>
>>>>>> String query = "PREFIX owl:<
>>>>>> http://www.w3.org/2002/07/owl#**** <http://www.w3.org/2002/07/owl#**>>\n"
>>>>>> +
>>>>>> "PREFIX geopos:<
>>>>>> http://www.w3.org/2003/01/geo/****wgs84_pos#<http://www.w3.org/2003/01/geo/**wgs84_pos#>
>>>>>> <http://www.w3.**org/2003/01/geo/wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>
>>>>>> >>\**n"
>>>>>>
>>>>>> +
>>>>>> "select ?s ?lat ?long where {\n" +
>>>>>> " SERVICE<
>>>>>> http://fuseki.mooo.com/lgd-****dbpedia/sparql<http://fuseki.mooo.com/lgd-**dbpedia/sparql>
>>>>>> <http://fuseki.**mooo.com/lgd-dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>
>>>>>> >>
>>>>>>
>>>>>> {\n" +
>>>>>> " select * where {\n" +
>>>>>> " ?s owl:sameAs ?geo .\n" +
>>>>>> " FILTER
>>>>>> regex(str(?geo),\"dbpedia\")\****n" +
>>>>>>
>>>>>> " } *limit 20000* \n" +
>>>>>>
>>>>>> " }\n" +
>>>>>> " service<
>>>>>> http://fuseki.mooo.com/****dbpedia-geo/sparql<http://fuseki.mooo.com/**dbpedia-geo/sparql>
>>>>>> <http://**fuseki.mooo.com/dbpedia-geo/**sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>
>>>>>> >>
>>>>>>
>>>>>> {\n" +
>>>>>> " select * where{\n" +
>>>>>> " ?geo geopos:lat ?lat ;\n" +
>>>>>> " geopos:long ?long .\n" +
>>>>>> " } \n" +
>>>>>> " } \n" +
>>>>>> "} \n";
>>>>>>
>>>>>>
>>>>>> What can I do to fix the problem? It always occurs when the query is
>>>>>> run,
>>>>>> even when the Fuseki Sparql Endpoint is running locally.
>>>>>> I did not observe any problem related to the network. The problem
>>>>>> persists
>>>>>> when the code (query) is executed in different hosts.
>>>>>> I am using Jena classes from "fuseki-server.jar" in the client
>>>>>> classpath.
>>>>>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
>>>>>> The full stack trace is this:
>>>>>>
>>>>>> Exception in thread "main" HttpException:
>>>>>> java.net.****NoRouteToHostException: Cannot assign requested address:
>>>>>> java.net.****NoRouteToHostException: Cannot assign requested address
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****http.HttpQuery.execGet(**
>>>>>> HttpQuery.java:216)
>>>>>> at com.hp.hpl.jena.sparql.engine.****http.HttpQuery.exec(**
>>>>>> HttpQuery.**
>>>>>> java:157)
>>>>>> at com.hp.hpl.jena.sparql.engine.****http.Service.exec(Service.**
>>>>>> java:71)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****main.iterator.**
>>>>>> QueryIterService.nextStage(****QueryIterService.java:53)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIterRepeatApply.**
>>>>>> makeNextStage(****QueryIterRepeatApply.java:113)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIterRepeatApply.**
>>>>>> hasNextBinding(****QueryIterRepeatApply.java:65)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIterConvert.**
>>>>>> hasNextBinding(****QueryIterConvert.java:65)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIteratorWrapper.**
>>>>>> hasNextBinding(****QueryIteratorWrapper.java:40)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.**QueryIteratorWrapper.**
>>>>>> hasNextBinding(****QueryIteratorWrapper.java:40)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****iterator.QueryIteratorBase.****
>>>>>> hasNext(QueryIteratorBase.****java:108)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****ResultSetStream.hasNext(**
>>>>>>
>>>>>> ResultSetStream.java:72)
>>>>>> at Query.main(Query.java:34)
>>>>>> Caused by: java.net.****NoRouteToHostException: Cannot assign
>>>>>> requested
>>>>>> address
>>>>>> at java.net.PlainSocketImpl.****socketConnect(Native Method)
>>>>>> at
>>>>>> java.net.****AbstractPlainSocketImpl.****doConnect(**
>>>>>> AbstractPlainSocketImpl.java:****327)
>>>>>> at
>>>>>> java.net.****AbstractPlainSocketImpl.****connectToAddress(**
>>>>>> AbstractPlainSocketImpl.java:****193)
>>>>>> at
>>>>>> java.net.****AbstractPlainSocketImpl.****connect(**
>>>>>> AbstractPlainSocketImpl.java:****180)
>>>>>> at java.net.SocksSocketImpl.****connect(SocksSocketImpl.java:****384)
>>>>>> at java.net.Socket.connect(****Socket.java:546)
>>>>>> at java.net.Socket.connect(****Socket.java:495)
>>>>>> at sun.net.NetworkClient.****doConnect(NetworkClient.java:****178)
>>>>>> at sun.net.www.http.HttpClient.****openServer(HttpClient.java:****
>>>>>> 409)
>>>>>> at sun.net.www.http.HttpClient.****openServer(HttpClient.java:****
>>>>>> 530)
>>>>>> at sun.net.www.http.HttpClient.<****init>(HttpClient.java:240)
>>>>>> at sun.net.www.http.HttpClient.****New(HttpClient.java:321)
>>>>>> at sun.net.www.http.HttpClient.****New(HttpClient.java:338)
>>>>>> at
>>>>>> sun.net.www.protocol.http.****HttpURLConnection.****
>>>>>> getNewHttpClient(**
>>>>>> HttpURLConnection.java:935)
>>>>>> at
>>>>>> sun.net.www.protocol.http.****HttpURLConnection.****plainConnect(**
>>>>>> HttpURLConnection.java:876)
>>>>>> at
>>>>>> sun.net.www.protocol.http.****HttpURLConnection.connect(**
>>>>>> HttpURLConnection.java:801)
>>>>>> at
>>>>>> com.hp.hpl.jena.sparql.engine.****http.HttpQuery.execGet(**
>>>>>>
>>>>>> HttpQuery.java:199)
>>>>>> ... 14 more
>>>>>>
>>>>>> Best regards,
>>>>>> Regis.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Error in Federated Query
Posted by Andy Seaborne <an...@apache.org>.
Regis,
What is fuseki.mooo.com? Are you running that server?
500 means the server is unhappy -- the server log file may give further
details.
I don't know why you are seeing large numbers of connections - the
client code uses vanilla java.net HTTP code so it should reuse
connections or close them (at least in the ARQ 2.9.0).
I'm not familiar with NginX as a proxy. ARQ does try to execute more
specific SERVICE requests that use one vague one - sometimes the right
thing to do, sometimes not. The engine can't tell.
We did recently (post 2.9.0 release - available in 2.9.1-SNAPSHOT)
change the SERVICE handling code. If you say "SERVICE SILENT" it will
read all the results from a call before moving on - that might help.
Andy
On 08/04/12 14:00, Regis Pires Magalhães wrote:
> This code can be used to see the HttpException (500 Server Error) remotely
> (only 18 results found before the exception occurs):
>
> import com.hp.hpl.jena.query.QueryExecution;
> import com.hp.hpl.jena.query.QueryExecutionFactory;
> import com.hp.hpl.jena.query.QueryFactory;
> import com.hp.hpl.jena.query.QuerySolution;
> import com.hp.hpl.jena.query.ResultSet;
> import com.hp.hpl.jena.rdf.model.ModelFactory;
> import com.hp.hpl.jena.sparql.core.DatasetImpl;
>
> public class Query {
> public static void main(String[] args) {
> String query = "PREFIX owl:<http://www.w3.org/2002/07/owl#>\n"
> +
> "PREFIX geopos:<
> http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
> "select ?s ?lat ?long where {\n" +
> " SERVICE
> <http://*fuseki.mooo.com:8080*/lgd-dbpedia/sparql>
> {\n" +
> " select * where {\n" +
> " ?s owl:sameAs ?geo .\n" +
> " FILTER regex(str(?geo),\"dbpedia\")\n"
> +
> " } \n" +
> " }\n" +
> " SERVICE
> <http://*fuseki.mooo.com:8080*/dbpedia-geo/sparql>
> {\n" +
> " select * where{\n" +
> " ?geo geopos:lat ?lat ;\n" +
> " geopos:long ?long .\n" +
> " } \n" +
> " } \n" +
> "} \n";
>
> long start = System.currentTimeMillis();
> QueryExecution exec =
> QueryExecutionFactory.create(QueryFactory.create(query),
> new
> DatasetImpl(ModelFactory.createDefaultModel()));
> ResultSet rs = exec.execSelect();
> int n = 0;
> while (rs.hasNext()) {
> rs.next();
> System.out.print(++n + " ");
> }
> long time = System.currentTimeMillis() - start;
> System.out.println(n + " results found in " + time + "ms.");
> }
> }
>
>
> Abraços,
> Regis.
>
>
> 2012/4/8 Regis Pires Magalhães<re...@gmail.com>
>
>> Andy,
>> The exception is always the same NoRouteToHostException and we the only
>> proxy we use between the client and the Fuseki server is a reverse proxy
>> (NginX).
>>
>> I've also observed that the number of network connections with the server
>> increases from 0 to a bit more than 28000, when the exception
>> occurs. Almost all of the connections are TIME_WAIT.
>>
>> When I try a direct connection to Fuseki (even locally using 127.0.0.1 as
>> the address to the Sparql Endpoint) I only get 17 results (the number of
>> connections increases from 0 to 212) and the exception changes to:
>>
>> Exception in thread "main" HttpException: HttpException: 500 Server Error
>> : HttpException: 500 Server Error
>> at
>> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execCommon(HttpQuery.java:375)
>> at com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:202)
>> at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:157)
>> at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:71)
>> at
>> com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:65)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
>> at Query.main(Query.java:42)
>> Caused by: HttpException: 500 Server Error
>> at
>> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execCommon(HttpQuery.java:320)
>> ... 15 more
>>
>> How could I get all the results?
>>
>> Regards,
>> Regis.
>>
>>
>>
>> On Sun, Apr 8, 2012 at 7:38 AM, Andy Seaborne<an...@apache.org> wrote:
>>
>>> On 08/04/12 01:30, Regis Pires Magalhăes wrote:
>>>
>>>> I've got a sucessful result when executing *outside* our intranet (LAN).
>>>>
>>>> All the 43016 results were returned.
>>>> It seems that there are so many requests (when executing in the Local
>>>> Network) that the server or the client can not properly
>>>> handle them. Suggestions? I could not receive more than 5000 results when
>>>> querying inside the LAN. Results come much faster, but the exception
>>>> always
>>>> occurs.
>>>>
>>>> Regis.
>>>>
>>>
>>> Is it the same exception? NoRouteToHostException isn't usually a sign of
>>> overload - it's a sign that the JVM inside the firewall can't contact the
>>> machine outside. Your proxy may be doing funny stuff though.
>>>
>>> Do you have proxy settings given to the JVM?it should work.
>>>
>>> -Dhttp.proxyHost=...
>>> -Dhttp.proxyPort=...
>>>
>>> or
>>> -DsocksProxyHost=...
>>> -DsocksProxyPort=...
>>>
>>> Andy
>>>
>>>
>>>>
>>>> 2012/4/7 Regis Pires Magalhăes<re...@gmail.com>
>>>>>
>>>>
>>>>
>>>>> I get a "java.net.**NoRouteToHostException: Cannot assign requested
>>>>> address"
>>>>> exception when I execute que following code:
>>>>>
>>>>> import com.hp.hpl.jena.query.**QueryExecution;
>>>>> import com.hp.hpl.jena.query.**QueryExecutionFactory;
>>>>> import com.hp.hpl.jena.query.**QueryFactory;
>>>>> import com.hp.hpl.jena.query.**QuerySolution;
>>>>> import com.hp.hpl.jena.query.**ResultSet;
>>>>> import com.hp.hpl.jena.rdf.model.**ModelFactory;
>>>>> import com.hp.hpl.jena.sparql.core.**DatasetImpl;
>>>>>
>>>>> public class Query {
>>>>> public static void main(String[] args) {
>>>>> String query = "PREFIX owl:<
>>>>> http://www.w3.org/2002/07/owl#**>\n" +
>>>>> "PREFIX geopos:<
>>>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>>>> +
>>>>> "select ?s ?lat ?long where {\n" +
>>>>> " SERVICE<
>>>>> http://fuseki.mooo.com/lgd-**dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>>
>>>>> {\n" +
>>>>> " select * where {\n" +
>>>>> " ?s owl:sameAs ?geo .\n" +
>>>>> " FILTER
>>>>> regex(str(?geo),\"dbpedia\")\**n" +
>>>>> " } \n" +
>>>>> " }\n" +
>>>>> " SERVICE<
>>>>> http://fuseki.mooo.com/**dbpedia-geo/sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>>
>>>>> {\n" +
>>>>> " select * where{\n" +
>>>>> " ?geo geopos:lat ?lat ;\n" +
>>>>> " geopos:long ?long .\n" +
>>>>> " } \n" +
>>>>> " } \n" +
>>>>> "} \n";
>>>>>
>>>>> long start = System.currentTimeMillis();
>>>>> QueryExecution exec =
>>>>> QueryExecutionFactory.create(**QueryFactory.create(query),
>>>>> new
>>>>> DatasetImpl(ModelFactory.**createDefaultModel()));
>>>>> ResultSet rs = exec.execSelect();
>>>>> int n = 0;
>>>>> while (rs.hasNext()) {
>>>>> rs.next();
>>>>> System.out.print(++n + " ");
>>>>> }
>>>>> long time = System.currentTimeMillis() - start;
>>>>> System.out.println(n + " results found in " + time +
>>>>> "ms.");
>>>>> }
>>>>> }
>>>>>
>>>>> The same query runs perfectly when a do limit in the first SERVICE
>>>>> select:
>>>>>
>>>>> String query = "PREFIX owl:<
>>>>> http://www.w3.org/2002/07/owl#**>\n" +
>>>>> "PREFIX geopos:<
>>>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>>>> +
>>>>> "select ?s ?lat ?long where {\n" +
>>>>> " SERVICE<
>>>>> http://fuseki.mooo.com/lgd-**dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>>
>>>>> {\n" +
>>>>> " select * where {\n" +
>>>>> " ?s owl:sameAs ?geo .\n" +
>>>>> " FILTER
>>>>> regex(str(?geo),\"dbpedia\")\**n" +
>>>>> " } *limit 20000* \n" +
>>>>>
>>>>> " }\n" +
>>>>> " service<
>>>>> http://fuseki.mooo.com/**dbpedia-geo/sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>>
>>>>> {\n" +
>>>>> " select * where{\n" +
>>>>> " ?geo geopos:lat ?lat ;\n" +
>>>>> " geopos:long ?long .\n" +
>>>>> " } \n" +
>>>>> " } \n" +
>>>>> "} \n";
>>>>>
>>>>>
>>>>> What can I do to fix the problem? It always occurs when the query is
>>>>> run,
>>>>> even when the Fuseki Sparql Endpoint is running locally.
>>>>> I did not observe any problem related to the network. The problem
>>>>> persists
>>>>> when the code (query) is executed in different hosts.
>>>>> I am using Jena classes from "fuseki-server.jar" in the client
>>>>> classpath.
>>>>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
>>>>> The full stack trace is this:
>>>>>
>>>>> Exception in thread "main" HttpException:
>>>>> java.net.**NoRouteToHostException: Cannot assign requested address:
>>>>> java.net.**NoRouteToHostException: Cannot assign requested address
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>>>> HttpQuery.java:216)
>>>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.exec(HttpQuery.**
>>>>> java:157)
>>>>> at com.hp.hpl.jena.sparql.engine.**http.Service.exec(Service.**
>>>>> java:71)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**main.iterator.**
>>>>> QueryIterService.nextStage(**QueryIterService.java:53)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>>>> makeNextStage(**QueryIterRepeatApply.java:113)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>>>> hasNextBinding(**QueryIterRepeatApply.java:65)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>>> hasNext(QueryIteratorBase.**java:108)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterConvert.**
>>>>> hasNextBinding(**QueryIterConvert.java:65)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>>> hasNext(QueryIteratorBase.**java:108)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>>> hasNext(QueryIteratorBase.**java:108)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>>> hasNext(QueryIteratorBase.**java:108)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**ResultSetStream.hasNext(**
>>>>> ResultSetStream.java:72)
>>>>> at Query.main(Query.java:34)
>>>>> Caused by: java.net.**NoRouteToHostException: Cannot assign requested
>>>>> address
>>>>> at java.net.PlainSocketImpl.**socketConnect(Native Method)
>>>>> at
>>>>> java.net.**AbstractPlainSocketImpl.**doConnect(**
>>>>> AbstractPlainSocketImpl.java:**327)
>>>>> at
>>>>> java.net.**AbstractPlainSocketImpl.**connectToAddress(**
>>>>> AbstractPlainSocketImpl.java:**193)
>>>>> at
>>>>> java.net.**AbstractPlainSocketImpl.**connect(**
>>>>> AbstractPlainSocketImpl.java:**180)
>>>>> at java.net.SocksSocketImpl.**connect(SocksSocketImpl.java:**384)
>>>>> at java.net.Socket.connect(**Socket.java:546)
>>>>> at java.net.Socket.connect(**Socket.java:495)
>>>>> at sun.net.NetworkClient.**doConnect(NetworkClient.java:**178)
>>>>> at sun.net.www.http.HttpClient.**openServer(HttpClient.java:**409)
>>>>> at sun.net.www.http.HttpClient.**openServer(HttpClient.java:**530)
>>>>> at sun.net.www.http.HttpClient.<**init>(HttpClient.java:240)
>>>>> at sun.net.www.http.HttpClient.**New(HttpClient.java:321)
>>>>> at sun.net.www.http.HttpClient.**New(HttpClient.java:338)
>>>>> at
>>>>> sun.net.www.protocol.http.**HttpURLConnection.**getNewHttpClient(**
>>>>> HttpURLConnection.java:935)
>>>>> at
>>>>> sun.net.www.protocol.http.**HttpURLConnection.**plainConnect(**
>>>>> HttpURLConnection.java:876)
>>>>> at
>>>>> sun.net.www.protocol.http.**HttpURLConnection.connect(**
>>>>> HttpURLConnection.java:801)
>>>>> at
>>>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>>>> HttpQuery.java:199)
>>>>> ... 14 more
>>>>>
>>>>> Best regards,
>>>>> Regis.
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
This code can be used to see the HttpException (500 Server Error) remotely
(only 18 results found before the exception occurs):
import com.hp.hpl.jena.query.QueryExecution;
import com.hp.hpl.jena.query.QueryExecutionFactory;
import com.hp.hpl.jena.query.QueryFactory;
import com.hp.hpl.jena.query.QuerySolution;
import com.hp.hpl.jena.query.ResultSet;
import com.hp.hpl.jena.rdf.model.ModelFactory;
import com.hp.hpl.jena.sparql.core.DatasetImpl;
public class Query {
public static void main(String[] args) {
String query = "PREFIX owl: <http://www.w3.org/2002/07/owl#>\n"
+
"PREFIX geopos: <
http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
"select ?s ?lat ?long where {\n" +
" SERVICE
<http://*fuseki.mooo.com:8080*/lgd-dbpedia/sparql>
{\n" +
" select * where {\n" +
" ?s owl:sameAs ?geo .\n" +
" FILTER regex(str(?geo),\"dbpedia\")\n"
+
" } \n" +
" }\n" +
" SERVICE
<http://*fuseki.mooo.com:8080*/dbpedia-geo/sparql>
{\n" +
" select * where{\n" +
" ?geo geopos:lat ?lat ;\n" +
" geopos:long ?long .\n" +
" } \n" +
" } \n" +
"} \n";
long start = System.currentTimeMillis();
QueryExecution exec =
QueryExecutionFactory.create(QueryFactory.create(query),
new
DatasetImpl(ModelFactory.createDefaultModel()));
ResultSet rs = exec.execSelect();
int n = 0;
while (rs.hasNext()) {
rs.next();
System.out.print(++n + " ");
}
long time = System.currentTimeMillis() - start;
System.out.println(n + " results found in " + time + "ms.");
}
}
Abraços,
Regis.
2012/4/8 Regis Pires Magalhães <re...@gmail.com>
> Andy,
> The exception is always the same NoRouteToHostException and we the only
> proxy we use between the client and the Fuseki server is a reverse proxy
> (NginX).
>
> I've also observed that the number of network connections with the server
> increases from 0 to a bit more than 28000, when the exception
> occurs. Almost all of the connections are TIME_WAIT.
>
> When I try a direct connection to Fuseki (even locally using 127.0.0.1 as
> the address to the Sparql Endpoint) I only get 17 results (the number of
> connections increases from 0 to 212) and the exception changes to:
>
> Exception in thread "main" HttpException: HttpException: 500 Server Error
> : HttpException: 500 Server Error
> at
> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execCommon(HttpQuery.java:375)
> at com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:202)
> at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:157)
> at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:71)
> at
> com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:65)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
> at Query.main(Query.java:42)
> Caused by: HttpException: 500 Server Error
> at
> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execCommon(HttpQuery.java:320)
> ... 15 more
>
> How could I get all the results?
>
> Regards,
> Regis.
>
>
>
> On Sun, Apr 8, 2012 at 7:38 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 08/04/12 01:30, Regis Pires Magalhăes wrote:
>>
>>> I've got a sucessful result when executing *outside* our intranet (LAN).
>>>
>>> All the 43016 results were returned.
>>> It seems that there are so many requests (when executing in the Local
>>> Network) that the server or the client can not properly
>>> handle them. Suggestions? I could not receive more than 5000 results when
>>> querying inside the LAN. Results come much faster, but the exception
>>> always
>>> occurs.
>>>
>>> Regis.
>>>
>>
>> Is it the same exception? NoRouteToHostException isn't usually a sign of
>> overload - it's a sign that the JVM inside the firewall can't contact the
>> machine outside. Your proxy may be doing funny stuff though.
>>
>> Do you have proxy settings given to the JVM?it should work.
>>
>> -Dhttp.proxyHost=...
>> -Dhttp.proxyPort=...
>>
>> or
>> -DsocksProxyHost=...
>> -DsocksProxyPort=...
>>
>> Andy
>>
>>
>>>
>>> 2012/4/7 Regis Pires Magalhăes<re...@gmail.com>
>>> >
>>>
>>>
>>>> I get a "java.net.**NoRouteToHostException: Cannot assign requested
>>>> address"
>>>> exception when I execute que following code:
>>>>
>>>> import com.hp.hpl.jena.query.**QueryExecution;
>>>> import com.hp.hpl.jena.query.**QueryExecutionFactory;
>>>> import com.hp.hpl.jena.query.**QueryFactory;
>>>> import com.hp.hpl.jena.query.**QuerySolution;
>>>> import com.hp.hpl.jena.query.**ResultSet;
>>>> import com.hp.hpl.jena.rdf.model.**ModelFactory;
>>>> import com.hp.hpl.jena.sparql.core.**DatasetImpl;
>>>>
>>>> public class Query {
>>>> public static void main(String[] args) {
>>>> String query = "PREFIX owl:<
>>>> http://www.w3.org/2002/07/owl#**>\n" +
>>>> "PREFIX geopos:<
>>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>>> +
>>>> "select ?s ?lat ?long where {\n" +
>>>> " SERVICE<
>>>> http://fuseki.mooo.com/lgd-**dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>>
>>>> {\n" +
>>>> " select * where {\n" +
>>>> " ?s owl:sameAs ?geo .\n" +
>>>> " FILTER
>>>> regex(str(?geo),\"dbpedia\")\**n" +
>>>> " } \n" +
>>>> " }\n" +
>>>> " SERVICE<
>>>> http://fuseki.mooo.com/**dbpedia-geo/sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>>
>>>> {\n" +
>>>> " select * where{\n" +
>>>> " ?geo geopos:lat ?lat ;\n" +
>>>> " geopos:long ?long .\n" +
>>>> " } \n" +
>>>> " } \n" +
>>>> "} \n";
>>>>
>>>> long start = System.currentTimeMillis();
>>>> QueryExecution exec =
>>>> QueryExecutionFactory.create(**QueryFactory.create(query),
>>>> new
>>>> DatasetImpl(ModelFactory.**createDefaultModel()));
>>>> ResultSet rs = exec.execSelect();
>>>> int n = 0;
>>>> while (rs.hasNext()) {
>>>> rs.next();
>>>> System.out.print(++n + " ");
>>>> }
>>>> long time = System.currentTimeMillis() - start;
>>>> System.out.println(n + " results found in " + time +
>>>> "ms.");
>>>> }
>>>> }
>>>>
>>>> The same query runs perfectly when a do limit in the first SERVICE
>>>> select:
>>>>
>>>> String query = "PREFIX owl:<
>>>> http://www.w3.org/2002/07/owl#**>\n" +
>>>> "PREFIX geopos:<
>>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>>> +
>>>> "select ?s ?lat ?long where {\n" +
>>>> " SERVICE<
>>>> http://fuseki.mooo.com/lgd-**dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>>
>>>> {\n" +
>>>> " select * where {\n" +
>>>> " ?s owl:sameAs ?geo .\n" +
>>>> " FILTER
>>>> regex(str(?geo),\"dbpedia\")\**n" +
>>>> " } *limit 20000* \n" +
>>>>
>>>> " }\n" +
>>>> " service<
>>>> http://fuseki.mooo.com/**dbpedia-geo/sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>>
>>>> {\n" +
>>>> " select * where{\n" +
>>>> " ?geo geopos:lat ?lat ;\n" +
>>>> " geopos:long ?long .\n" +
>>>> " } \n" +
>>>> " } \n" +
>>>> "} \n";
>>>>
>>>>
>>>> What can I do to fix the problem? It always occurs when the query is
>>>> run,
>>>> even when the Fuseki Sparql Endpoint is running locally.
>>>> I did not observe any problem related to the network. The problem
>>>> persists
>>>> when the code (query) is executed in different hosts.
>>>> I am using Jena classes from "fuseki-server.jar" in the client
>>>> classpath.
>>>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
>>>> The full stack trace is this:
>>>>
>>>> Exception in thread "main" HttpException:
>>>> java.net.**NoRouteToHostException: Cannot assign requested address:
>>>> java.net.**NoRouteToHostException: Cannot assign requested address
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>>> HttpQuery.java:216)
>>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.exec(HttpQuery.**
>>>> java:157)
>>>> at com.hp.hpl.jena.sparql.engine.**http.Service.exec(Service.**
>>>> java:71)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**main.iterator.**
>>>> QueryIterService.nextStage(**QueryIterService.java:53)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>>> makeNextStage(**QueryIterRepeatApply.java:113)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>>> hasNextBinding(**QueryIterRepeatApply.java:65)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterConvert.**
>>>> hasNextBinding(**QueryIterConvert.java:65)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>>> hasNext(QueryIteratorBase.**java:108)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**ResultSetStream.hasNext(**
>>>> ResultSetStream.java:72)
>>>> at Query.main(Query.java:34)
>>>> Caused by: java.net.**NoRouteToHostException: Cannot assign requested
>>>> address
>>>> at java.net.PlainSocketImpl.**socketConnect(Native Method)
>>>> at
>>>> java.net.**AbstractPlainSocketImpl.**doConnect(**
>>>> AbstractPlainSocketImpl.java:**327)
>>>> at
>>>> java.net.**AbstractPlainSocketImpl.**connectToAddress(**
>>>> AbstractPlainSocketImpl.java:**193)
>>>> at
>>>> java.net.**AbstractPlainSocketImpl.**connect(**
>>>> AbstractPlainSocketImpl.java:**180)
>>>> at java.net.SocksSocketImpl.**connect(SocksSocketImpl.java:**384)
>>>> at java.net.Socket.connect(**Socket.java:546)
>>>> at java.net.Socket.connect(**Socket.java:495)
>>>> at sun.net.NetworkClient.**doConnect(NetworkClient.java:**178)
>>>> at sun.net.www.http.HttpClient.**openServer(HttpClient.java:**409)
>>>> at sun.net.www.http.HttpClient.**openServer(HttpClient.java:**530)
>>>> at sun.net.www.http.HttpClient.<**init>(HttpClient.java:240)
>>>> at sun.net.www.http.HttpClient.**New(HttpClient.java:321)
>>>> at sun.net.www.http.HttpClient.**New(HttpClient.java:338)
>>>> at
>>>> sun.net.www.protocol.http.**HttpURLConnection.**getNewHttpClient(**
>>>> HttpURLConnection.java:935)
>>>> at
>>>> sun.net.www.protocol.http.**HttpURLConnection.**plainConnect(**
>>>> HttpURLConnection.java:876)
>>>> at
>>>> sun.net.www.protocol.http.**HttpURLConnection.connect(**
>>>> HttpURLConnection.java:801)
>>>> at
>>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>>> HttpQuery.java:199)
>>>> ... 14 more
>>>>
>>>> Best regards,
>>>> Regis.
>>>>
>>>>
>>>>
>>>
>>
>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
Andy,
The exception is always the same NoRouteToHostException and we the only
proxy we use between the client and the Fuseki server is a reverse proxy
(NginX).
I've also observed that the number of network connections with the server
increases from 0 to a bit more than 28000, when the exception
occurs. Almost all of the connections are TIME_WAIT.
When I try a direct connection to Fuseki (even locally using 127.0.0.1 as
the address to the Sparql Endpoint) I only get 17 results (the number of
connections increases from 0 to 212) and the exception changes to:
Exception in thread "main" HttpException: HttpException: 500 Server Error
: HttpException: 500 Server Error
at
com.hp.hpl.jena.sparql.engine.http.HttpQuery.execCommon(HttpQuery.java:375)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:202)
at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:157)
at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:71)
at
com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:65)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
at
com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
at Query.main(Query.java:42)
Caused by: HttpException: 500 Server Error
at
com.hp.hpl.jena.sparql.engine.http.HttpQuery.execCommon(HttpQuery.java:320)
... 15 more
How could I get all the results?
Regards,
Regis.
On Sun, Apr 8, 2012 at 7:38 AM, Andy Seaborne <an...@apache.org> wrote:
> On 08/04/12 01:30, Regis Pires Magalhăes wrote:
>
>> I've got a sucessful result when executing *outside* our intranet (LAN).
>>
>> All the 43016 results were returned.
>> It seems that there are so many requests (when executing in the Local
>> Network) that the server or the client can not properly
>> handle them. Suggestions? I could not receive more than 5000 results when
>> querying inside the LAN. Results come much faster, but the exception
>> always
>> occurs.
>>
>> Regis.
>>
>
> Is it the same exception? NoRouteToHostException isn't usually a sign of
> overload - it's a sign that the JVM inside the firewall can't contact the
> machine outside. Your proxy may be doing funny stuff though.
>
> Do you have proxy settings given to the JVM?it should work.
>
> -Dhttp.proxyHost=...
> -Dhttp.proxyPort=...
>
> or
> -DsocksProxyHost=...
> -DsocksProxyPort=...
>
> Andy
>
>
>>
>> 2012/4/7 Regis Pires Magalhăes<re...@gmail.com>
>> >
>>
>>
>>> I get a "java.net.**NoRouteToHostException: Cannot assign requested
>>> address"
>>> exception when I execute que following code:
>>>
>>> import com.hp.hpl.jena.query.**QueryExecution;
>>> import com.hp.hpl.jena.query.**QueryExecutionFactory;
>>> import com.hp.hpl.jena.query.**QueryFactory;
>>> import com.hp.hpl.jena.query.**QuerySolution;
>>> import com.hp.hpl.jena.query.**ResultSet;
>>> import com.hp.hpl.jena.rdf.model.**ModelFactory;
>>> import com.hp.hpl.jena.sparql.core.**DatasetImpl;
>>>
>>> public class Query {
>>> public static void main(String[] args) {
>>> String query = "PREFIX owl:<
>>> http://www.w3.org/2002/07/owl#**>\n" +
>>> "PREFIX geopos:<
>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>> +
>>> "select ?s ?lat ?long where {\n" +
>>> " SERVICE<
>>> http://fuseki.mooo.com/lgd-**dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>>
>>> {\n" +
>>> " select * where {\n" +
>>> " ?s owl:sameAs ?geo .\n" +
>>> " FILTER
>>> regex(str(?geo),\"dbpedia\")\**n" +
>>> " } \n" +
>>> " }\n" +
>>> " SERVICE<
>>> http://fuseki.mooo.com/**dbpedia-geo/sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>>
>>> {\n" +
>>> " select * where{\n" +
>>> " ?geo geopos:lat ?lat ;\n" +
>>> " geopos:long ?long .\n" +
>>> " } \n" +
>>> " } \n" +
>>> "} \n";
>>>
>>> long start = System.currentTimeMillis();
>>> QueryExecution exec =
>>> QueryExecutionFactory.create(**QueryFactory.create(query),
>>> new
>>> DatasetImpl(ModelFactory.**createDefaultModel()));
>>> ResultSet rs = exec.execSelect();
>>> int n = 0;
>>> while (rs.hasNext()) {
>>> rs.next();
>>> System.out.print(++n + " ");
>>> }
>>> long time = System.currentTimeMillis() - start;
>>> System.out.println(n + " results found in " + time +
>>> "ms.");
>>> }
>>> }
>>>
>>> The same query runs perfectly when a do limit in the first SERVICE
>>> select:
>>>
>>> String query = "PREFIX owl:<
>>> http://www.w3.org/2002/07/owl#**>\n" +
>>> "PREFIX geopos:<
>>> http://www.w3.org/2003/01/geo/**wgs84_pos#<http://www.w3.org/2003/01/geo/wgs84_pos#>>\n"
>>> +
>>> "select ?s ?lat ?long where {\n" +
>>> " SERVICE<
>>> http://fuseki.mooo.com/lgd-**dbpedia/sparql<http://fuseki.mooo.com/lgd-dbpedia/sparql>>
>>> {\n" +
>>> " select * where {\n" +
>>> " ?s owl:sameAs ?geo .\n" +
>>> " FILTER
>>> regex(str(?geo),\"dbpedia\")\**n" +
>>> " } *limit 20000* \n" +
>>>
>>> " }\n" +
>>> " service<
>>> http://fuseki.mooo.com/**dbpedia-geo/sparql<http://fuseki.mooo.com/dbpedia-geo/sparql>>
>>> {\n" +
>>> " select * where{\n" +
>>> " ?geo geopos:lat ?lat ;\n" +
>>> " geopos:long ?long .\n" +
>>> " } \n" +
>>> " } \n" +
>>> "} \n";
>>>
>>>
>>> What can I do to fix the problem? It always occurs when the query is run,
>>> even when the Fuseki Sparql Endpoint is running locally.
>>> I did not observe any problem related to the network. The problem
>>> persists
>>> when the code (query) is executed in different hosts.
>>> I am using Jena classes from "fuseki-server.jar" in the client classpath.
>>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
>>> The full stack trace is this:
>>>
>>> Exception in thread "main" HttpException:
>>> java.net.**NoRouteToHostException: Cannot assign requested address:
>>> java.net.**NoRouteToHostException: Cannot assign requested address
>>> at
>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>> HttpQuery.java:216)
>>> at com.hp.hpl.jena.sparql.engine.**http.HttpQuery.exec(HttpQuery.**
>>> java:157)
>>> at com.hp.hpl.jena.sparql.engine.**http.Service.exec(Service.**java:71)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**main.iterator.**
>>> QueryIterService.nextStage(**QueryIterService.java:53)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>> makeNextStage(**QueryIterRepeatApply.java:113)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterRepeatApply.**
>>> hasNextBinding(**QueryIterRepeatApply.java:65)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIterConvert.**
>>> hasNextBinding(**QueryIterConvert.java:65)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorWrapper.**
>>> hasNextBinding(**QueryIteratorWrapper.java:40)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**iterator.QueryIteratorBase.**
>>> hasNext(QueryIteratorBase.**java:108)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**ResultSetStream.hasNext(**
>>> ResultSetStream.java:72)
>>> at Query.main(Query.java:34)
>>> Caused by: java.net.**NoRouteToHostException: Cannot assign requested
>>> address
>>> at java.net.PlainSocketImpl.**socketConnect(Native Method)
>>> at
>>> java.net.**AbstractPlainSocketImpl.**doConnect(**
>>> AbstractPlainSocketImpl.java:**327)
>>> at
>>> java.net.**AbstractPlainSocketImpl.**connectToAddress(**
>>> AbstractPlainSocketImpl.java:**193)
>>> at
>>> java.net.**AbstractPlainSocketImpl.**connect(**
>>> AbstractPlainSocketImpl.java:**180)
>>> at java.net.SocksSocketImpl.**connect(SocksSocketImpl.java:**384)
>>> at java.net.Socket.connect(**Socket.java:546)
>>> at java.net.Socket.connect(**Socket.java:495)
>>> at sun.net.NetworkClient.**doConnect(NetworkClient.java:**178)
>>> at sun.net.www.http.HttpClient.**openServer(HttpClient.java:**409)
>>> at sun.net.www.http.HttpClient.**openServer(HttpClient.java:**530)
>>> at sun.net.www.http.HttpClient.<**init>(HttpClient.java:240)
>>> at sun.net.www.http.HttpClient.**New(HttpClient.java:321)
>>> at sun.net.www.http.HttpClient.**New(HttpClient.java:338)
>>> at
>>> sun.net.www.protocol.http.**HttpURLConnection.**getNewHttpClient(**
>>> HttpURLConnection.java:935)
>>> at
>>> sun.net.www.protocol.http.**HttpURLConnection.**plainConnect(**
>>> HttpURLConnection.java:876)
>>> at
>>> sun.net.www.protocol.http.**HttpURLConnection.connect(**
>>> HttpURLConnection.java:801)
>>> at
>>> com.hp.hpl.jena.sparql.engine.**http.HttpQuery.execGet(**
>>> HttpQuery.java:199)
>>> ... 14 more
>>>
>>> Best regards,
>>> Regis.
>>>
>>>
>>>
>>
>
Re: Error in Federated Query
Posted by Andy Seaborne <an...@apache.org>.
On 08/04/12 01:30, Regis Pires Magalhães wrote:
> I've got a sucessful result when executing *outside* our intranet (LAN).
> All the 43016 results were returned.
> It seems that there are so many requests (when executing in the Local
> Network) that the server or the client can not properly
> handle them. Suggestions? I could not receive more than 5000 results when
> querying inside the LAN. Results come much faster, but the exception always
> occurs.
>
> Regis.
Is it the same exception? NoRouteToHostException isn't usually a sign
of overload - it's a sign that the JVM inside the firewall can't contact
the machine outside. Your proxy may be doing funny stuff though.
Do you have proxy settings given to the JVM?it should work.
-Dhttp.proxyHost=...
-Dhttp.proxyPort=...
or
-DsocksProxyHost=...
-DsocksProxyPort=...
Andy
>
>
> 2012/4/7 Regis Pires Magalhães<re...@gmail.com>
>
>>
>> I get a "java.net.NoRouteToHostException: Cannot assign requested address"
>> exception when I execute que following code:
>>
>> import com.hp.hpl.jena.query.QueryExecution;
>> import com.hp.hpl.jena.query.QueryExecutionFactory;
>> import com.hp.hpl.jena.query.QueryFactory;
>> import com.hp.hpl.jena.query.QuerySolution;
>> import com.hp.hpl.jena.query.ResultSet;
>> import com.hp.hpl.jena.rdf.model.ModelFactory;
>> import com.hp.hpl.jena.sparql.core.DatasetImpl;
>>
>> public class Query {
>> public static void main(String[] args) {
>> String query = "PREFIX owl:<
>> http://www.w3.org/2002/07/owl#>\n" +
>> "PREFIX geopos:<
>> http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
>> "select ?s ?lat ?long where {\n" +
>> " SERVICE<
>> http://fuseki.mooo.com/lgd-dbpedia/sparql> {\n" +
>> " select * where {\n" +
>> " ?s owl:sameAs ?geo .\n" +
>> " FILTER
>> regex(str(?geo),\"dbpedia\")\n" +
>> " } \n" +
>> " }\n" +
>> " SERVICE<
>> http://fuseki.mooo.com/dbpedia-geo/sparql> {\n" +
>> " select * where{\n" +
>> " ?geo geopos:lat ?lat ;\n" +
>> " geopos:long ?long .\n" +
>> " } \n" +
>> " } \n" +
>> "} \n";
>>
>> long start = System.currentTimeMillis();
>> QueryExecution exec =
>> QueryExecutionFactory.create(QueryFactory.create(query),
>> new
>> DatasetImpl(ModelFactory.createDefaultModel()));
>> ResultSet rs = exec.execSelect();
>> int n = 0;
>> while (rs.hasNext()) {
>> rs.next();
>> System.out.print(++n + " ");
>> }
>> long time = System.currentTimeMillis() - start;
>> System.out.println(n + " results found in " + time +
>> "ms.");
>> }
>> }
>>
>> The same query runs perfectly when a do limit in the first SERVICE select:
>>
>> String query = "PREFIX owl:<
>> http://www.w3.org/2002/07/owl#>\n" +
>> "PREFIX geopos:<
>> http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
>> "select ?s ?lat ?long where {\n" +
>> " SERVICE<
>> http://fuseki.mooo.com/lgd-dbpedia/sparql> {\n" +
>> " select * where {\n" +
>> " ?s owl:sameAs ?geo .\n" +
>> " FILTER
>> regex(str(?geo),\"dbpedia\")\n" +
>> " } *limit 20000* \n" +
>> " }\n" +
>> " service<
>> http://fuseki.mooo.com/dbpedia-geo/sparql> {\n" +
>> " select * where{\n" +
>> " ?geo geopos:lat ?lat ;\n" +
>> " geopos:long ?long .\n" +
>> " } \n" +
>> " } \n" +
>> "} \n";
>>
>>
>> What can I do to fix the problem? It always occurs when the query is run,
>> even when the Fuseki Sparql Endpoint is running locally.
>> I did not observe any problem related to the network. The problem persists
>> when the code (query) is executed in different hosts.
>> I am using Jena classes from "fuseki-server.jar" in the client classpath.
>> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
>> The full stack trace is this:
>>
>> Exception in thread "main" HttpException:
>> java.net.NoRouteToHostException: Cannot assign requested address:
>> java.net.NoRouteToHostException: Cannot assign requested address
>> at
>> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:216)
>> at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:157)
>> at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:71)
>> at
>> com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:65)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
>> at
>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
>> at
>> com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
>> at Query.main(Query.java:34)
>> Caused by: java.net.NoRouteToHostException: Cannot assign requested address
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
>> at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
>> at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
>> at java.net.Socket.connect(Socket.java:546)
>> at java.net.Socket.connect(Socket.java:495)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
>> at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
>> at sun.net.www.http.HttpClient.New(HttpClient.java:321)
>> at sun.net.www.http.HttpClient.New(HttpClient.java:338)
>> at
>> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
>> at
>> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
>> at
>> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
>> at
>> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:199)
>> ... 14 more
>>
>> Best regards,
>> Regis.
>>
>>
>
Re: Error in Federated Query
Posted by Regis Pires Magalhães <re...@gmail.com>.
I've got a sucessful result when executing *outside* our intranet (LAN).
All the 43016 results were returned.
It seems that there are so many requests (when executing in the Local
Network) that the server or the client can not properly
handle them. Suggestions? I could not receive more than 5000 results when
querying inside the LAN. Results come much faster, but the exception always
occurs.
Regis.
2012/4/7 Regis Pires Magalhães <re...@gmail.com>
>
> I get a "java.net.NoRouteToHostException: Cannot assign requested address"
> exception when I execute que following code:
>
> import com.hp.hpl.jena.query.QueryExecution;
> import com.hp.hpl.jena.query.QueryExecutionFactory;
> import com.hp.hpl.jena.query.QueryFactory;
> import com.hp.hpl.jena.query.QuerySolution;
> import com.hp.hpl.jena.query.ResultSet;
> import com.hp.hpl.jena.rdf.model.ModelFactory;
> import com.hp.hpl.jena.sparql.core.DatasetImpl;
>
> public class Query {
> public static void main(String[] args) {
> String query = "PREFIX owl: <
> http://www.w3.org/2002/07/owl#>\n" +
> "PREFIX geopos: <
> http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
> "select ?s ?lat ?long where {\n" +
> " SERVICE <
> http://fuseki.mooo.com/lgd-dbpedia/sparql> {\n" +
> " select * where {\n" +
> " ?s owl:sameAs ?geo .\n" +
> " FILTER
> regex(str(?geo),\"dbpedia\")\n" +
> " } \n" +
> " }\n" +
> " SERVICE <
> http://fuseki.mooo.com/dbpedia-geo/sparql> {\n" +
> " select * where{\n" +
> " ?geo geopos:lat ?lat ;\n" +
> " geopos:long ?long .\n" +
> " } \n" +
> " } \n" +
> "} \n";
>
> long start = System.currentTimeMillis();
> QueryExecution exec =
> QueryExecutionFactory.create(QueryFactory.create(query),
> new
> DatasetImpl(ModelFactory.createDefaultModel()));
> ResultSet rs = exec.execSelect();
> int n = 0;
> while (rs.hasNext()) {
> rs.next();
> System.out.print(++n + " ");
> }
> long time = System.currentTimeMillis() - start;
> System.out.println(n + " results found in " + time +
> "ms.");
> }
> }
>
> The same query runs perfectly when a do limit in the first SERVICE select:
>
> String query = "PREFIX owl: <
> http://www.w3.org/2002/07/owl#>\n" +
> "PREFIX geopos: <
> http://www.w3.org/2003/01/geo/wgs84_pos#>\n" +
> "select ?s ?lat ?long where {\n" +
> " SERVICE <
> http://fuseki.mooo.com/lgd-dbpedia/sparql> {\n" +
> " select * where {\n" +
> " ?s owl:sameAs ?geo .\n" +
> " FILTER
> regex(str(?geo),\"dbpedia\")\n" +
> " } *limit 20000* \n" +
> " }\n" +
> " service <
> http://fuseki.mooo.com/dbpedia-geo/sparql> {\n" +
> " select * where{\n" +
> " ?geo geopos:lat ?lat ;\n" +
> " geopos:long ?long .\n" +
> " } \n" +
> " } \n" +
> "} \n";
>
>
> What can I do to fix the problem? It always occurs when the query is run,
> even when the Fuseki Sparql Endpoint is running locally.
> I did not observe any problem related to the network. The problem persists
> when the code (query) is executed in different hosts.
> I am using Jena classes from "fuseki-server.jar" in the client classpath.
> I've also tested different Fuseki jar versions: 0.2.1, 0.2.2.
> The full stack trace is this:
>
> Exception in thread "main" HttpException:
> java.net.NoRouteToHostException: Cannot assign requested address:
> java.net.NoRouteToHostException: Cannot assign requested address
> at
> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:216)
> at com.hp.hpl.jena.sparql.engine.http.HttpQuery.exec(HttpQuery.java:157)
> at com.hp.hpl.jena.sparql.engine.http.Service.exec(Service.java:71)
> at
> com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterService.nextStage(QueryIterService.java:53)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.makeNextStage(QueryIterRepeatApply.java:113)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:65)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:65)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
> at
> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:108)
> at
> com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)
> at Query.main(Query.java:34)
> Caused by: java.net.NoRouteToHostException: Cannot assign requested address
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
> at java.net.Socket.connect(Socket.java:546)
> at java.net.Socket.connect(Socket.java:495)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:178)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
> at sun.net.www.http.HttpClient.New(HttpClient.java:321)
> at sun.net.www.http.HttpClient.New(HttpClient.java:338)
> at
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:935)
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:876)
> at
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:801)
> at
> com.hp.hpl.jena.sparql.engine.http.HttpQuery.execGet(HttpQuery.java:199)
> ... 14 more
>
> Best regards,
> Regis.
>
>