You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jena.apache.org by "ting668 (via GitHub)" <gi...@apache.org> on 2026/10/07 04:40:42 UTC
[I] FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on [jena]
ting668 opened a new issue, #4295:
URL: https://github.com/apache/jena/issues/4295
### Version
6.2.0
### What happened?
**Reporter:** ting668
## Environment
- Apache Jena Version: `6.2.0`
- Java Version: `21.0.12+8-LTS`
## Description
While testing Apache Jena using an intuitive approach inspired by attribute constraints, I found that the default `optFilterEquality` optimizer rewrites `FILTER(sameTerm(?x, literal))` into a constant substitution plus assignment, which changes the answer: rows that `sameTerm` must reject are returned, and the returned `?x` binding is replaced by the constant.
`sameTerm` is term equality (SPARQL 1.1 §17.4.1.8: "same RDF term"), not value equality. `"01"^^xsd:integer` and `"1"^^xsd:integer` are different RDF terms with the same value, so `sameTerm(?x, "1"^^xsd:integer)` accepts only the `"1"` term. With the optimizer on (the default), Jena returns both rows and reports `?x` as the integer `1` for both — i.e. the filter was answered by value equality after substituting the constant into the pattern. With the optimizer disabled via the context symbol `http://jena.apache.org/ARQ#optFilterEquality=false`, the same query returns only the `s2` row, so the two configurations disagree about the query result.
## How to Reproduce and Expected Behavior
**Note:** The queries below are a **minimized, simplified example** reproducing the bug.
Data (default graph, Turtle):
```turtle
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX : <http://x/>
:s1 :p "01"^^xsd:integer .
:s2 :p "1"^^xsd:integer .
```
Buggy query (default configuration, no reasoner):
```sparql
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX : <http://x/>
SELECT ?s ?x WHERE { ?s :p ?x FILTER(sameTerm(?x, "1"^^xsd:integer)) }
```
Expected behavior (`sameTerm` is term equality):
```text
----------------------------
| s | x |
============================
| :s2 | "1"^^xsd:integer |
----------------------------
```
Actual behavior (Jena 6.2.0, defaults):
```text
----------------------------
| s | x |
============================
| :s2 | 1 |
| :s1 | 1 |
----------------------------
```
Two defects are visible at once: the `s1` row (`"01"^^xsd:integer`, a different term) passes the filter, and `?x` no longer carries the term that matched — it is the substituted constant.
Control: the same query run with the optimizer disabled (context symbol `http://jena.apache.org/ARQ#optFilterEquality=false`) returns exactly the expected single `s2` row, so the difference is introduced by the rewrite, not by the pattern matching.
The `E_Equals` (value equality) case in the same optimizer contains a safety analysis that declines the rewrite for literals whose "lexical forms are not 1-1 with values"; the `E_SameTerm` case substitutes the constant for any literal without that check, after which the substituted pattern matches by value equality and the original term is lost from the solution.
### Relevant output and stacktrace
```shell
```
### Are you interested in making a pull request?
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
rvesse commented on issue #4295:
URL: https://github.com/apache/jena/issues/4295#issuecomment-6034130148
You need to describe what you mean by Jena defaults here.
Running your query against the standalone command line `sparql` tool i.e.
```bash
$ sparql --file ~/Downloads/sameTerm.sparql --data ~/Downloads/sameTerm.ttl
-----------
| s | x |
===========
| :s2 | 1 |
-----------
```
Produces the expected result.
My suspicion is that maybe you are using TDB here? Per documentation TDB [inlines values](https://jena.apache.org/documentation/tdb/architecture.html#inline-values) so those two objects would be the same node if the data is loaded into TDB. This is a performance trade off and known documented behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
rvesse closed issue #4295: FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on
URL: https://github.com/apache/jena/issues/4295
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on [jena]
Posted by "ting668 (via GitHub)" <gi...@apache.org>.
ting668 commented on issue #4295:
URL: https://github.com/apache/jena/issues/4295#issuecomment-6035326157
<html>
<body>
<!--StartFragment--><html><head></head><body><p><span>Thanks for checking. You are right that my wording “Jena defaults” was too broad. This was embedded execution through </span><code dir="ltr">QueryExecutionFactory.create(query, model)</code><span> over </span><code dir="ltr">ModelFactory.createDefaultModel()</code><span>, backed by </span><code dir="ltr">GraphMemValue</code><span>, without TDB. I rechecked it using the official Jena 6.2.0 release jar and Java 21.0.11+10-LTS.</span></p><p><span>For the original data—</span><code dir="ltr">:s1 :p "01"^^xsd:integer</code><span> and </span><code dir="ltr">:s2 :p "1"^^xsd:integer</code><span>—and the query </span><code dir="ltr">SELECT ?s ?x WHERE { ?s :p ?x FILTER(sameTerm(?x, "1"^^xsd:integer)) }</code><span>, the results on the same Model are:</span></p><div><div><div>
Configuration | Result
-- | --
Defaults | s1 and s2, both with ?x = "1"^^xsd:integer
ARQ.optFilterEquality=false | s2 only
ARQ.optimization=false | s2 only
</div></div></div><p><span>Two controls clarify the difference: without the filter, </span><code dir="ltr">s1</code><span> retains </span><code dir="ltr">?x = "01"^^xsd:integer</code><span>; projecting </span><code dir="ltr">sameTerm(?x, "1"^^xsd:integer)</code><span> returns false for </span><code dir="ltr">s1</code><span> and true for </span><code dir="ltr">s2</code><span>. Thus the lexical distinction survives loading, and evaluating the expression directly distinguishes the terms.</span></p><p><span>The optimized algebra substitutes the constant into the triple pattern and assigns it back to </span><code dir="ltr">?x</code><span>, removing the explicit </span><code dir="ltr">sameTerm</code><span> check. The result then depends on the Model’s value-based pattern matching. This explains both the extra row and the replacement of the original </span><code dir="ltr">?x</code><span> binding with the constant.</span></p><p><span>As an additional control, adding </span><code dir="lt
r">:s3 :p "1.0"^^xsd:decimal</code><span> produces three rows with defaults, all with </span><code dir="ltr">?x</code><span> bound to the integer constant. Disabling either optimizer setting again returns only </span><code dir="ltr">s2</code><span>.</span></p><p><code dir="ltr">ModelFactory.createModelSameTerm()</code><span>, </span><code dir="ltr">DatasetFactory.create()</code><span>, TxnMem, and the standalone CLI return only </span><code dir="ltr">s2</code><span> in all three optimizer configurations for the original data, agreeing with your result.</span></p><p><span>I understand the connection with </span><a href="https://github.com/apache/jena/issues/4255"><span><span><span>#4255</span></span></span></a><span> and the compatibility rationale. Could you clarify whether this also covers the same explicit </span><code dir="ltr">sameTerm</code><span> query returning different results on an unchanged Model when only </span><code dir="ltr">optFilterEquality</code><span> changes? Tha
t would help determine whether a brief documentation note or an optimizer applicability guard would be useful. If you consider this covered by the existing compatibility behavior, I’m happy for the issue to be linked to #4255 or closed on that basis. Thanks again for looking into it.</span></p></body></html><!--EndFragment-->
</body>
</html>Thanks for checking. You are right that my wording “Jena defaults” was too broad. This was embedded execution through `QueryExecutionFactory.create(query, model)` over `ModelFactory.createDefaultModel()`, backed by `GraphMemValue`, without TDB. I rechecked it using the official Jena 6.2.0 release jar and Java 21.0.11+10-LTS.
For the original data—`:s1 :p "01"^^xsd:integer` and `:s2 :p "1"^^xsd:integer`—and the query `SELECT ?s ?x WHERE { ?s :p ?x FILTER(sameTerm(?x, "1"^^xsd:integer)) }`, the results on the same Model are:
| Configuration | Result |
| --- | --- |
| Defaults | `s1` and `s2`, both with `?x = "1"^^xsd:integer` |
| `ARQ.optFilterEquality=false` | `s2` only |
| `ARQ.optimization=false` | `s2` only |
Two controls clarify the difference: without the filter, `s1` retains `?x = "01"^^xsd:integer`; projecting `sameTerm(?x, "1"^^xsd:integer)` returns false for `s1` and true for `s2`. Thus the lexical distinction survives loading, and evaluating the expression directly distinguishes the terms.
The optimized algebra substitutes the constant into the triple pattern and assigns it back to `?x`, removing the explicit `sameTerm` check. The result then depends on the Model’s value-based pattern matching. This explains both the extra row and the replacement of the original `?x` binding with the constant.
As an additional control, adding `:s3 :p "1.0"^^xsd:decimal` produces three rows with defaults, all with `?x` bound to the integer constant. Disabling either optimizer setting again returns only `s2`.
`ModelFactory.createModelSameTerm()`, `DatasetFactory.create()`, TxnMem, and the standalone CLI return only `s2` in all three optimizer configurations for the original data, agreeing with your result.
I understand the connection with [[#4255](https://github.com/apache/jena/issues/4255)](https://github.com/apache/jena/issues/4255) and the compatibility rationale. Could you clarify whether this also covers the same explicit `sameTerm` query returning different results on an unchanged Model when only `optFilterEquality` changes? That would help determine whether a brief documentation note or an optimizer applicability guard would be useful. If you consider this covered by the existing compatibility behavior, I’m happy for the issue to be linked to #4255 or closed on that basis. Thanks again for looking into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
rvesse commented on issue #4295:
URL: https://github.com/apache/jena/issues/4295#issuecomment-6037776081
> Thanks for checking. You are right that my wording “Jena defaults” was too broad. This was embedded execution through `QueryExecutionFactory.create(query, model)` over `ModelFactory.createDefaultModel()`, backed by `GraphMemValue`, without TDB. I rechecked it using the official Jena 6.2.0 release jar and Java 21.0.11+10-LTS.
So this is the key detail, ultimately you are using `GraphMemValue` which has value match semantics. So when the optimiser rewrites the filter to a direct BGP scanning for that value it's `GraphMemValue` that is returning the faulty values.
You could get the same behaviour by rewriting the query as follows:
```sparql
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX : <http://x/>
SELECT ?s WHERE { ?s :p "1"^^xsd:integer }
```
Therefore the optimisation is not the problem here, it's the choice of the underlying storage implementation.
Most real world Jena usage for SPARQL querying is via the `DatasetGraph` APIs which as my testing shows is not affected because the newer `Graph`/`DatasetGraph` APIs don't have the legacy same-value BGP matching behaviour of the `Model` API. If you actually want to test "defaults" then use `DatasetGraphFactory.create()` and work with the `DatasetGraph` API which does not have as much of the legacy baggage as `Model` does.
The `Model` API is one of the oldest APIs in Jena with substantial inertia, changing it creates real impact for the established user base. A few weird edge cases (that most users are never going to care about) is not sufficient to make a breaking change for users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] FILTER(sameTerm(?x, literal)) returns extra rows and rewrites ?x when the optFilterEquality optimizer is on [jena]
Posted by "rvesse (via GitHub)" <gi...@apache.org>.
rvesse commented on issue #4295:
URL: https://github.com/apache/jena/issues/4295#issuecomment-6034167293
Possible duplicate/closely related to #4255
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org