You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@jena.apache.org by "Aklakan (via GitHub)" <gi...@apache.org> on 2025/08/29 17:14:35 UTC

[PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Aklakan opened a new pull request, #3405:
URL: https://github.com/apache/jena/pull/3405

   GitHub issue resolved #3404
   
   Pull request Description: Proposal to restore lexical form of decimals stored in inlined TDB2 NodeIds using their plain string form rather than the canonical form. There are tests for decimals that assert that for a Node `n` round-tripping works, i.e. `nodeId(n) == nodeId(extract(nodeId(n))`.
   
   ----
   
    - [x] Tests are included.
    - [x] Commits have been squashed to remove intermediate development commit messages.
    - [x] Key commit messages start with the issue number (GH-xxxx)
   
   By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the [Contributor's Agreement](https://www.apache.org/licenses/contributor-agreements.html).
   
   ----
   
   See the [Apache Jena "Contributing" guide](https://github.com/apache/jena/blob/main/CONTRIBUTING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3245981822

   > I just updated the commit message for the PR - other than that I think its done(?)
   
   From your side - yes.
   
   I want to investigate the consequences, such as range which seems to be changed  e.g. 15000000000000 (15e12), and conversely, very small numbers.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3266833408

   Thanks!
   
   Closjng this and continuing on #3428.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3266313993

   OK - I'll put this on a branch in the Jena GH repo.
   
   It can stay on the branch until jena6. It's not an area that is likely to cause conflicts up to jena6.
   
   (I have some code tidy to apply as well)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3246328819

   I don't have need for separate modes as I have upgraded my data to the canonical form already. I was just thinking about how other users could fix their data in case they run into this issue. But perhaps just requiring a reload with jena 6 is the easiest way.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on code in PR #3405:
URL: https://github.com/apache/jena/pull/3405#discussion_r2312003069


##########
jena-tdb2/src/test/java/org/apache/jena/tdb2/store/value/TestNodeIdInline.java:
##########
@@ -120,6 +120,29 @@ public class TestNodeIdInline
     @Test public void nodeId_decimal_8()
     { test("0.00000001", NodeFactory.createLiteralDT("0.00000001", XSDDatatype.XSDdecimal)); }
 
+    @Test public void nodeId_decimal_9()
+    { test("'18'^^xsd:decimal",
+           NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal));
+    }
+
+    @Test public void nodeId_decimal_10()
+    { test("'18.'^^xsd:decimal", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_11()
+    { test("18.0", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_20()
+    { testNodeIdRoundtrip(NodeValue.makeDecimal("18").asNode()); }

Review Comment:
   Better to use `NodeFactory.createLiteralDT`, posisbly via a new  `testNodeIdRoundtripDecimal(String)`.
   
   `NodeValue.makeDecimal` may have other effects; in fact, adding it into the round triple steps might be a good thing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3246097544

   Just my two cents: The exponential notation of decimals in BigDecimal seems to be a mere feature of certain "toString" functions. `Decimal.toPlainString` does not generate exponent fields.
   
   https://www.w3.org/TR/xmlschema-2/#decimal-lexical-representation
   
   > 3.2.3.1 Lexical representation
   decimal has a lexical representation consisting of a finite-length sequence of decimal digits (#x30-#x39) separated by a period as a decimal indicator. An optional leading sign is allowed. If the sign is omitted, "+" is assumed. Leading and trailing zeroes are optional. If the fractional part is zero, the period and following zero(es) can be omitted. For example: -1.23, 12678967.543233, +100000.00, 210.
   
   > 3.2.3.2 Canonical representation
   The canonical representation for decimal is defined by prohibiting certain options from the [Lexical representation (ยง3.2.3.1)](https://www.w3.org/TR/xmlschema-2/#decimal-lexical-representation). Specifically, the preceding optional "+" sign is prohibited. The decimal point is required. Leading and trailing zeroes are prohibited subject to the following: there must be at least one digit to the right and to the left of the decimal point which may be a zero. 
   
   https://docs.oracle.com/en/java/javase/24/docs/api/java.base/java/math/BigDecimal.html#toPlainString()
   > public [String](https://docs.oracle.com/en/java/javase/24/docs/api/java.base/java/lang/String.html) toPlainString()
   Returns a string representation of this BigDecimal without an exponent field. For values with a positive scale, the number of digits to the right of the decimal point is used to indicate scale. For values with a zero or negative scale, the resulting string is generated as if the value were converted to a numerically equal value with zero scale and as if all the trailing zeros of the zero scale value were present in the result. The entire string is prefixed by a minus sign character '-' ('\u002D') if the unscaled value is less than zero. No sign character is prefixed if the unscaled value is zero or positive.
   
   
   As a side note, perhaps it is desired in TDB2 to use expose non-canonical Decimals directly with `toPlainString` instead of `canonicalDecimalStrNoIntegerDot`. The implication is that legacy stored decimals would be exposed without the trailing `.0`. However, legacy data could then be upgraded with this sparql statement (at least in theory):
   `DELETE { ?s ?p ?o } INSERT { ?s ?p ?o } WHERE { ?s ?p ?o FILTER(datatype(?o) = <http://www.w3.org/2001/XMLSchema#decimal>) }`
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3240104043

   I just updated the commit message for the PR - other than that I think its done(?)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3239261229

   Inserts/Deletes/Lookups quads with non-canonical decimals now works reliably :partying_face: 
   
   As a side note, with the PR applied, our (streaming) sameAs reasoner now also works as expected - no more exceptions about lookups failing to detect the lexicographically least physical quad in the store upon which to enrich the graph.find stream with all the sameAs inferences. We created this reasoner plugin because for this specific use case, the performance of the owl reasoner and the rule engine was too bad  and this approach let's one selectively enable reasoning for specific parts of a query.
   
   SameAs inferences using `SERVICE <sameAs:> (filtered to decimals): https://api.triplydb.com/s/OI_O18TlJ
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3266419533

   Pulled and rebased to the current main on my working repo (a safer place than apache/jena ... just in case of a misstep).
   
   https://github.com/afs/jena/pull/new/decimal56
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on code in PR #3405:
URL: https://github.com/apache/jena/pull/3405#discussion_r2312003069


##########
jena-tdb2/src/test/java/org/apache/jena/tdb2/store/value/TestNodeIdInline.java:
##########
@@ -120,6 +120,29 @@ public class TestNodeIdInline
     @Test public void nodeId_decimal_8()
     { test("0.00000001", NodeFactory.createLiteralDT("0.00000001", XSDDatatype.XSDdecimal)); }
 
+    @Test public void nodeId_decimal_9()
+    { test("'18'^^xsd:decimal",
+           NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal));
+    }
+
+    @Test public void nodeId_decimal_10()
+    { test("'18.'^^xsd:decimal", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_11()
+    { test("18.0", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_20()
+    { testNodeIdRoundtrip(NodeValue.makeDecimal("18").asNode()); }

Review Comment:
   Better to use `NodeFactory.createLiteralDT`, possibly via a new  `testNodeIdRoundtripDecimal(String)`.
   
   `NodeValue.makeDecimal` may have other effects; in fact, adding it into the round triple steps might be a good thing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs closed pull request #3405: GH-3404: Canonicalize decimals during inlining for TDB2.
URL: https://github.com/apache/jena/pull/3405


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3239462676

   > Is this after reloading the data?
   
   I needed to reload the offending graphs (I used 5.6.0-SNAPSHOT)  - the TDB2 version used to load the data initially was around 4.8.0 or 4.9.0.
   
   > Is that a typo for "foo"@en-US
   Right, the issue is not about the order of the components - but the use of lower case only. I was able to reproduce my issue with an older version of Jena:
   
   ```
   ./tdb2.tdbupdate --loc /tmp/tdb 'INSERT DATA { <urn:s> <urn:p> "foo"@en-gb }'
   ./tdb2.tdbquery --loc /tmp/tdb 'SELECT * { { BIND(1 AS ?id) <urn:s> <urn:p> "foo"@en-gb } UNION { BIND(2 AS ?id) <urn:s> <urn:p> "foo"@en-GB } }'
   ```
   
   On my local 4.7.0 the query returns `{(?id=1)}` wheras on 5.6.0-SNAPSHOT I get `{(?id=1), (?id=2)}`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on code in PR #3405:
URL: https://github.com/apache/jena/pull/3405#discussion_r2312072033


##########
jena-tdb2/src/test/java/org/apache/jena/tdb2/store/value/TestNodeIdInline.java:
##########
@@ -120,6 +120,29 @@ public class TestNodeIdInline
     @Test public void nodeId_decimal_8()
     { test("0.00000001", NodeFactory.createLiteralDT("0.00000001", XSDDatatype.XSDdecimal)); }
 
+    @Test public void nodeId_decimal_9()
+    { test("'18'^^xsd:decimal",
+           NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal));
+    }
+
+    @Test public void nodeId_decimal_10()
+    { test("'18.'^^xsd:decimal", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_11()
+    { test("18.0", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_20()
+    { testNodeIdRoundtrip(NodeValue.makeDecimal("18").asNode()); }

Review Comment:
   The test cases are now updated to use `testNodeIdRoundtripDecimal(String)`,



##########
jena-tdb2/src/test/java/org/apache/jena/tdb2/store/value/TestNodeIdInline.java:
##########
@@ -120,6 +120,29 @@ public class TestNodeIdInline
     @Test public void nodeId_decimal_8()
     { test("0.00000001", NodeFactory.createLiteralDT("0.00000001", XSDDatatype.XSDdecimal)); }
 
+    @Test public void nodeId_decimal_9()
+    { test("'18'^^xsd:decimal",
+           NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal));
+    }
+
+    @Test public void nodeId_decimal_10()
+    { test("'18.'^^xsd:decimal", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_11()
+    { test("18.0", NodeFactory.createLiteralDT("18.0", XSDDatatype.XSDdecimal)); }
+
+    @Test public void nodeId_decimal_20()
+    { testNodeIdRoundtrip(NodeValue.makeDecimal("18").asNode()); }

Review Comment:
   The test cases are now updated to use `testNodeIdRoundtripDecimal(String)`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on code in PR #3405:
URL: https://github.com/apache/jena/pull/3405#discussion_r2311853498


##########
jena-arq/src/main/java/org/apache/jena/sparql/util/XSDNumUtils.java:
##########
@@ -147,6 +147,18 @@ public static String canonicalDecimalStrNoIntegerDot(BigDecimal bd) {
         return bd.stripTrailingZeros().toPlainString();
     }
 
+    /** Return a canonical decimal with a trailing ".0". */
+    public static BigDecimal canonicalDecimalWithDot(BigDecimal decimal) {

Review Comment:
   This new method is used on input after parsing the lexical form into a decimal - just before inlining. The Str Version ist used on output, after converting the decimal to string. I think that the Str Version could be removed in favor of Decimal.toPlainString() since the binary representation is now canonicalized on input.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on code in PR #3405:
URL: https://github.com/apache/jena/pull/3405#discussion_r2311853498


##########
jena-arq/src/main/java/org/apache/jena/sparql/util/XSDNumUtils.java:
##########
@@ -147,6 +147,18 @@ public static String canonicalDecimalStrNoIntegerDot(BigDecimal bd) {
         return bd.stripTrailingZeros().toPlainString();
     }
 
+    /** Return a canonical decimal with a trailing ".0". */
+    public static BigDecimal canonicalDecimalWithDot(BigDecimal decimal) {

Review Comment:
   This new method is used on input after parsing the lexical form into a decimal. The Str Version ist used on output, after converting the decimal to string. I think that the Str Version could be removed in favor of Decimal.toPlainString() since the binary representation is now canonicalized on input.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3238233722

   I added a pre-canonicalization step and both the existing tests and the now ones now pass.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3239425588

   > also works as expected
   
   Is this after reloading the data? Or new old - existing TDB2 database?
   
   > `"foo"^^us-EN`
   
   Is that a typo for `"foo"@en-US` (which is canonical). Jena keeps language tags in RFC case format, not lower case.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3246243793

   The `15e12` was just to make it easier to read. It's an illegal lexical form for xsd:decimal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on code in PR #3405:
URL: https://github.com/apache/jena/pull/3405#discussion_r2311849597


##########
jena-arq/src/main/java/org/apache/jena/sparql/util/XSDNumUtils.java:
##########
@@ -147,6 +147,18 @@ public static String canonicalDecimalStrNoIntegerDot(BigDecimal bd) {
         return bd.stripTrailingZeros().toPlainString();
     }
 
+    /** Return a canonical decimal with a trailing ".0". */
+    public static BigDecimal canonicalDecimalWithDot(BigDecimal decimal) {

Review Comment:
   How does this differ from `canonicalDecimalStrWithDot`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3240031592

   Jena 5.4.0 introduced `jena-langtag`.
   
   The RDF 1.1 spec says "language tags MAY be converted to lower case. The value space of language tags is always in lower case."  This leads to differences across systems (when are we in "value space" for a language tag?) but also differences between Jena dataset implementations.
   
   Now RDF 1.2 Concepts says "MUST be treated accordingly, that is, in a case-insensitive manner."
   
   Jena went with the algorithm in [RFC 5646 section 2.1.1](https://datatracker.ietf.org/doc/html/rfc5646#section-2.1.1), without regitry access, based on some previous user feedback. Language tag are now parsed or created as case-normalized following RFC 5646 form (lang is lowercase, region is uppercase) then comparison. The original language tag is not preserved.
   
   The WG did a survey of systems: it found systems providing lower case and systems providing the algorithm in RFC 5646 section 2.1.1. Jena went with the algorithm in RFC 5646 based on some previous user feedback.
   
   There is an [entry in the RDF 1.2 change log](https://www.w3.org/TR/rdf12-concepts/#:~:text=Implementations%20were%20previously%20allowed%20to%20normalize%20language%20tags%20to%20lower%20case%2C).
   
   For initial text direction, the strings are `ltr` or `rtl` and lower case is required.
   
   CHANGES for jena 5.4.0:
   
   > ==  New artifact
   > 
   > Jena 5.4.0 introduces a new module jena-langtag for language tag parsing 
   > in compliance with RFC 5646. Language tag validation is strengthened. 
   > When parsing, language tag violations are still treated as warnings.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Canonicalize decimals during inlining for TDB2. [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3246272699

   > The implication is that legacy stored non-canonical decimals would be exposed without the trailing `.0`
   
   BGP matching is by comparing the 64bit binary `NodeId` so there is a conversion on the way in from the query as well as one the way out.
   
   The `canonicalDecimalStrWithDot` works with Turtle short form.
   
   Jena v6 is, _hopefully_, the release after next. That would be a good time to make the change systematically and have a data reload.
   
   The final jena 5 release could have "current" and "improved" (for you, mainly!) with "improved" needing to be explicitly enabled. (The user is responsible for not mixing them.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org


Re: [PR] GH-3404: Restore lexical forms of TDB2 NodeId-decimals as plain strings. [jena]

Posted by "Aklakan (via GitHub)" <gi...@apache.org>.
Aklakan commented on PR #3405:
URL: https://github.com/apache/jena/pull/3405#issuecomment-3237749397

   Hm, I see that this PR causes TestNormalizationTDB2 to fail.
   I am not sure whether non-canonical decimals should be returned in their canonical form by TDB2 (the test cases suggest: yes) or whether this should only be the case when e.g. canonicalize literals is enabled on the parser.
   
   The alternative to make round-tripping work would be to canonicalize decimals during inlining. Then the NodeId that becomes stored would be that of the canonical decimal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org