You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jena.apache.org by "mpagni12 (via GitHub)" <gi...@apache.org> on 2025/02/19 15:44:15 UTC
[I] BIND clause slows down some queries [jena]
mpagni12 opened a new issue, #3021:
URL: https://github.com/apache/jena/issues/3021
### Version
5.3.0
### Question
Dear fuseki community,
I have observed several times with different queries, that the presence of a simple BIND clause can drastically slows down its execution.
For example, the following query
```{sparql}
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX reconx: <https://reconx.vital-it.ch/kg/>
PREFIX mnx: <https://rdf.metanetx.org/schema/>
SELECT
?mnet
?chem_1
?chem_1_label
?chem_2
?chem_2_label
WHERE{
{
SELECT DISTINCT ?mnet ?mnxm ?chem_1 ?chem_2
WHERE{
{
SELECT DISTINCT ?mnet ?mnxm ?chem_1 ?chem_2
WHERE{
BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
}
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_2
}
}
?chem_1 reconx:label ?chem_1_label .
?chem_2 reconx:label ?chem_2_label .
}
```
takes at least half an hour to execute on my local fuseki instance.
But, if I remove the BIND clause from the most inner WHERE clause, by inlining the subject:
```{sparql}
...
WHERE{
# BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
reconx:vmh_Recon reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
...
```
the query now executes in a couple of second!
Replacing the BIND clause with a VALUES clause also executes very slowly.
The same query executed on GraphDB populated with the same dataset takes a couple of second to execute, with no significant differences between the three variants (inline, BIND, VALUES)
I tend to prefer to use the syntax with the explicit BIND or VALUES clause, because in a complex query it permits to syntactically highlight the "input parameter". But currently, the price is too high. I wonder it has to do with the query optimiser.
This being reported, thanks a lot for maintaining fuseki which is a great *open-source* tool.
Marco
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2676311379
As a temporary workaround:
Removing `?mnet` from the inner-most SELECT DISTINCT is faster. The query then finds `?mnet` again.
The middle level `SELECT DISTINCT` gives the overall `SELECT DISTINCT ?mnet ?mnxm ?chem_1 ?chem_2`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2698939598
@mpagni12 - have you tried the temporary workaround?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2669737069
Hi @mpagni12
Looks like the optimizer is missing the best filter placement.
Could you try the following 2 patterns which add `{ }` to hint to the optimizer:
```
. . .
WHERE{
{
BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
}
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
. . .
```
and also
```
. . .
WHERE{
{
BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
{
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
}
. . .
```
Andy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2676282400
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2671672349
> The data are not yet officially released, but I can supply them to you by private email.
Sorry, I can't work with private data. I try to treat all bug reports the same. If I did this for one user, it would suggest it could be done for other users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2886040696
Great to see a solution coming. I tried previously to replace `BIND` with `VALUES` and didn't see any performance improvement. May I ask you, if your suggested patch will also improve `VALUES` performance when there is one or just a few different values?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2671174141
How can I join you privately?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2671048475
I have already attempted to unbundle the path, with no improvement in execution time. I can mention that the data structure behind the long property path is a dag, not a tree.
IMPORTANT: The isolated inner part executed as a stand-alone query is fast! Hence, the problem I report seems to be link to the graph pattern being executed in a sub-query. This greatly clarify the problem I think.
The data are not yet officially released, but I can supply them to you. However I guess that the problem should be easy to reproduce by introducing BIND clause in the innermost graph pattern of nested sub-queries.
The data are stored in TDB2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2702084973
Remove `?mnet` :
```sparql
WHERE{
{
SELECT DISTINCT ?mnet ?mnxm ?chem_1 ?chem_2
WHERE{
{
SELECT DISTINCT ?mnxm ?chem_1 ?chem_2 ## Removed ?mnet
WHERE{
BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
}
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_2
}
}
?chem_1 reconx:label ?chem_1_label .
?chem_2 reconx:label ?chem_2_label .
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2702055661
I don't really understand the above temporary workaround, sorry. In my hands, the previous solution
```sparql
WHERE{
# BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
reconx:vmh_Recon reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
```
remains the faster, and should stay so while the number of candidate `?mnet` (will) increase in the database.
Thank you for your help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2673836843
I understand, it makes sense. I tend to be very cautious by default, as I am often working with sensitive or unpublished data from other researchers. But it is not the case here.
Please find the [dump](https://reconx.vital-it.ch/data/reconx.ttl.gz) of the dataset I have used in my above testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2670951986
You can unbundle the path (although the query execution does that anyway)
```sparql
##?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnet reconx:reac ?V1 .
?V1 reconx:equaSource ?V2 .
?V2 reconx:part ?V3 .
?V3 reconx:spec ?V4 .
?V4 reconx:chem ?chem_1 .
```
Is the data publicly available?
Does the inner part:
```sparql
SELECT DISTINCT ?mnet ?mnxm ?chem_1 ?chem_2
WHERE{
BIND( reconx:vmh_Recon AS ?mnet ) # fix a model here to focus the results
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
```
behave the same way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2703204793
Yes it works, it is fast and it return the right number of rows.
But it is possibly specific to this query. I have other similar queries where I am quite sure this strategy won't work. Ultimately it may depend on how the intermediary nodes are organised (behind the long property paths), and on the overall database content, which will grow.
I found that assigning `?mnet` once in the innermost SELECT and propagating its value to the outermost SELECT, was quite readable (at least to me) and a rather systematic way to formulate such queries. It works as expected in GraphDB.
Maybe my question should be converted into a bug report.
Anyway, thanks a lot for all the time you spent digging into my problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2676234595
And there are 288 results?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2674066914
> Please find the [dump](https://reconx.vital-it.ch/data/reconx.ttl.gz) of the dataset I have used in my above testing.
Got it! 66,860,461 triples.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "mpagni12 (via GitHub)" <gi...@apache.org>.
mpagni12 commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2670845787
None of these help :-(
I have also tried, without success:
```sparql
. . .
WHERE{
{
SELECT ?mnet WHERE{ BIND( reconx:vmh_Recon AS ?mnet ) } # fix a model here to focus the results
}
?mnet reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem ?chem_1 .
?mnxm mnx:chemXref ?chem_1, ?chem_2
FILTER( STR( ?chem_1 ) < STR( ?chem_2 ))
}
. . .
```
I wonder that the property path `reconx:reac/reconx:equaSource/reconx:part/reconx:spec/reconx:chem` is executed in the wrong direction, unless `?mnet` is explicitely inlined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2671091040
> However I guess that the problem should be easy to reproduce by introducing BIND clause in the innermost graph pattern of nested sub-queries.
It's proving to be quite difficult to set up a simulation and be confident it illustrates the issue at your end. The optimizer plan doesn't look bad but there is a level below that which is more data shape sensitive. So the shape of the data appears to be a factor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] BIND clause slows down some queries [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #3021:
URL: https://github.com/apache/jena/issues/3021#issuecomment-2904997840
@mpagni12 Unfortunately, this is AI generated. It does not address your report.
This is a situation that open source projects have to deal with nowadays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org