You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jena.apache.org by "ddvlanck (via GitHub)" <gi...@apache.org> on 2025/11/18 15:26:48 UTC
[D] Performance issues on small dataset: any recommended configuration or tuning options? [jena]
GitHub user ddvlanck created a discussion: Performance issues on small dataset: any recommended configuration or tuning options?
Hi! We are currently evaluating Apache Jena Fuseki by loading a dataset and running a set of SPARQL queries against it.
However, I’m running into performance issues that seem unexpected for the dataset size.
⚠️ Issue
We are testing with a dataset containing **~16 million triples**, and we’ve configured a **60-second timeout** for query responses in our experiment setup.
All SELECT queries—ranging from those expected to return around **10 rows** up to those expected to return **~1M rows**—hit the timeout limit. Even the queries with very small expected result sets fail to return within 60 seconds.
The system running Apache Jena Fuseki has **128 GB RAM** available, so hardware limitations don’t appear to be the cause.
🧪 Setup
- Fuseki version: created a local Docker build using Apache Jena `5.6.0`
- Deployment: Docker
- Dataset size: 16,127,232 triples
- Query type: SELECT
- Operating system: Ubuntu 24.04.3 LTS
We updated the `JAVA_OPTIONS` to `-Xms90g -Xmx90g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+ParallelRefProcEnabled -XX:+AlwaysPreTouch -XX:+UseStringDeduplication`
💡 What we’re looking for
We’d like to understand whether there are configuration options, tuning parameters, or indexing strategies we should apply to improve query performance.
Any guidance on:
- Recommended settings for larger datasets
- Known performance limitations in Fuseki
- Configuration flags that influence query planning or execution
…would be greatly appreciated.
Thanks in advance!
GitHub link: https://github.com/apache/jena/discussions/3600
----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [D] Performance issues on small dataset: any recommended configuration or tuning options? [jena]
Posted by "ddvlanck (via GitHub)" <gi...@apache.org>.
GitHub user ddvlanck added a comment to the discussion: Performance issues on small dataset: any recommended configuration or tuning options?
Hi! 👋 Thanks a lot for the detailed explanation.
That’s very helpful — we’ll definitely lower the heap size accordingly to leave more room for off-heap memory. As for the queries, they all follow the same general structure:
```
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX tm: <https://movias.eu/ns/trafficMeasurements#>
PREFIX time: <https://movias.eu/ns/time#>
SELECT ?ts ?val
FROM {{GRAPH}}
WHERE {
?obs a tm:TrafficMeasurement ;
tm:TrafficMeasurement.measureLocation ?measureLocationId ;
tm:TrafficMeasurement.result ?val ;
tm:TrafficMeasurement.featureOfInterest ?featureOfInterest ;
tm:TrafficMeasurement.vehicleType ?vehicleType ;
tm:TrafficMeasurement.time ?timeNode .
?timeNode time:Time.dateTime ?ts .
FILTER (xsd:dateTime(?ts) >= "{{FROM}}"^^xsd:dateTime &&
xsd:dateTime(?ts) <= "{{TO}}"^^xsd:dateTime)
FILTER (?featureOfInterest = <https://data.vlaanderen.be/id/concept/VkmVerkeersKenmerkType/aantal>)
FILTER (?vehicleType = <https://data.vlaanderen.be/id/concept/VkmVoertuigType/fiets>)
}
ORDER BY ?ts
```
During testing, the query pattern stays the same — only the temporal range changes between runs, starting at 15 minutes and gradually expanding up to 10 years. I’ve also made a small sample dataset publicly available so you can reproduce the issue: https://github.com/ddvlanck/traffic-measurements-synthetic-data. These are some of the ranges for which the query failed:
- `2020-05-28T13:47:07` - `2020-05-28T14:02:07`
- `2015-05-12T02:55:38` - `2015-05-12T04:55:38`
- `2020-01-02T08:21:29` - `2021-01-02T08:21:29`
- `2015-01-01T00:14:35` - `2025-01-01T00:14:35`
I’ll take a closer look at the TDB optimiser documentation and use qparse to inspect the optimised algebra, as you suggested
GitHub link: https://github.com/apache/jena/discussions/3600#discussioncomment-15031496
----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [D] Performance issues on small dataset: any recommended configuration or tuning options? [jena]
Posted by "ddvlanck (via GitHub)" <gi...@apache.org>.
GitHub user ddvlanck edited a comment on the discussion: Performance issues on small dataset: any recommended configuration or tuning options?
Hi! Thanks a lot for the detailed explanation.
That’s very helpful — we’ll definitely lower the heap size accordingly to leave more room for off-heap memory. As for the queries, they all follow the same general structure:
```
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX tm: <https://movias.eu/ns/trafficMeasurements#>
PREFIX time: <https://movias.eu/ns/time#>
SELECT ?ts ?val
FROM {{GRAPH}}
WHERE {
?obs a tm:TrafficMeasurement ;
tm:TrafficMeasurement.measureLocation ?measureLocationId ;
tm:TrafficMeasurement.result ?val ;
tm:TrafficMeasurement.featureOfInterest ?featureOfInterest ;
tm:TrafficMeasurement.vehicleType ?vehicleType ;
tm:TrafficMeasurement.time ?timeNode .
?timeNode time:Time.dateTime ?ts .
FILTER (xsd:dateTime(?ts) >= "{{FROM}}"^^xsd:dateTime &&
xsd:dateTime(?ts) <= "{{TO}}"^^xsd:dateTime)
FILTER (?featureOfInterest = <https://data.vlaanderen.be/id/concept/VkmVerkeersKenmerkType/aantal>)
FILTER (?vehicleType = <https://data.vlaanderen.be/id/concept/VkmVoertuigType/fiets>)
}
ORDER BY ?ts
```
During testing, the query pattern stays the same — only the temporal range changes between runs, starting at 15 minutes and gradually expanding up to 10 years. I’ve also made a small sample dataset publicly available so you can reproduce the issue: https://github.com/ddvlanck/traffic-measurements-synthetic-data. These are some of the ranges for which the query failed:
- `2020-05-28T13:47:07` - `2020-05-28T14:02:07`
- `2015-05-12T02:55:38` - `2015-05-12T04:55:38`
- `2020-01-02T08:21:29` - `2021-01-02T08:21:29`
- `2015-01-01T00:14:35` - `2025-01-01T00:14:35`
I’ll take a closer look at the TDB optimiser documentation and use qparse to inspect the optimised algebra, as you suggested.
GitHub link: https://github.com/apache/jena/discussions/3600#discussioncomment-15031496
----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [D] Performance issues on small dataset: any recommended configuration or tuning options? [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
GitHub user rvesse edited a comment on the discussion: Performance issues on small dataset: any recommended configuration or tuning options?
> We updated the JAVA_OPTIONS to -Xms90g -Xmx90g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+ParallelRefProcEnabled -XX:+AlwaysPreTouch -XX:+UseStringDeduplication
Firstly don't set the heap size that high, most of the memory usage for TDB2 (assuming that is your database backend) is off-heap via memory mapped files so you want to leave plenty of memory for the OS to use for that.
I wouldn't typically go higher than 8GB heap at your scale (and even that is likely overkill unless your queries are very memory intensive)
https://jena.apache.org/documentation/tdb/faqs.html#java-heap
> Dataset size: 16,127,232 triples
Often performance is data driven so if you can share your dataset that's great, if not (e.g. it's commercially sensitive) can you generate a sample dataset that shows the same performance problems with your queries that you can share?
> Query type: SELECT
That's insufficient information for us to help you. `SELECT` is the main query type in SPARQL and has many possible clauses, without seeing some sample queries we can't tell whether your queries are the problem
> Configuration flags that influence query planning or execution
There's some documentation on TDB optimisation at https://jena.apache.org/documentation/tdb/optimizer.html
The default query optimiser code is at https://github.com/apache/jena/blob/main/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/OptimizerStd.java and you can see the various ARQ configuration symbols that turn specific optimisations on/off in the code there
You may also find it helpful to use the [`qparse`](https://jena.apache.org/documentation/query/cmds.html#arqqparse) command from ARQ to compare the unoptimised and optimised algebras to see if ARQ is actually able to optimise your queries at all
GitHub link: https://github.com/apache/jena/discussions/3600#discussioncomment-15011665
----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [D] Performance issues on small dataset: any recommended configuration or tuning options? [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
GitHub user rvesse added a comment to the discussion: Performance issues on small dataset: any recommended configuration or tuning options?
> We updated the JAVA_OPTIONS to -Xms90g -Xmx90g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+ParallelRefProcEnabled -XX:+AlwaysPreTouch -XX:+UseStringDeduplication
Firstly don't set the heap size that high, most of the memory usage for TDB2 (assuming that is your database backend) is off-heap via memory mapped files so you want to leave plenty of memory for the OS to use for that.
I wouldn't typically go higher than 8GB heap at your scale (and even that is likely overkill unless your queries are very memory intensive)
https://jena.apache.org/documentation/tdb/faqs.html#java-heap
> Dataset size: 16,127,232 triples
Often performance is data driven so if you can share your dataset that's great, if not (e.g. it's commercially sensitive) can you generate a sample dataset that shows the same performance problems with your queries that you can share
> Query type: SELECT
That's insufficient information for us to help you. `SELECT` is the main query type in SPARQL and has many possible clauses, without seeing some sample queries we can't tell whether your queries are the problem
> Configuration flags that influence query planning or execution
There's some documentation on TDB optimisation at https://jena.apache.org/documentation/tdb/optimizer.html
The default query optimiser code is at https://github.com/apache/jena/blob/main/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/OptimizerStd.java and you can see the various ARQ configuration symbols that turn specific optimisations on/off in the code there
GitHub link: https://github.com/apache/jena/discussions/3600#discussioncomment-15011665
----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [D] Performance issues on small dataset: any recommended configuration or tuning options? [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
GitHub user rvesse added a comment to the discussion: Performance issues on small dataset: any recommended configuration or tuning options?
You can also view the various algebra forms of your queries via our online [SPARQL Query Validator](https://sparql.org/query-validator.html)
GitHub link: https://github.com/apache/jena/discussions/3600#discussioncomment-15011695
----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org