You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "tagadearpit (via GitHub)" <gi...@apache.org> on 2026/08/23 12:11:42 UTC

[I] [Transform-V2] SQL lower and upper depend on the worker default locale [seatunnel]

tagadearpit opened a new issue, #11949:
URL: https://github.com/apache/seatunnel/issues/11949

   ### Description
   
   The SQL transform functions `lower()` and `upper()` call Java `String.toLowerCase()` and `String.toUpperCase()` without an explicit locale. As a result, their output depends on the default locale of the SeaTunnel worker JVM.
   
   For example, under the Turkish locale, `lower("I")` returns the dotless character `ı` and `upper("i")` returns `İ`, instead of the locale-independent SQL results `i` and `I`. A distributed job can therefore produce different transformed data depending on the host locale.
   
   ### Steps to reproduce
   
   1. Check out SeaTunnel `dev`.
   2. In `seatunnel-transforms-v2`, invoke the existing `StringFunction` methods with the JVM default locale set to Turkish:
   
   ```java
   Locale original = Locale.getDefault();
   try {
       Locale.setDefault(Locale.forLanguageTag("tr-TR"));
       System.out.println(StringFunction.lower(Collections.singletonList("I")));
       System.out.println(StringFunction.upper(Collections.singletonList("i")));
   } finally {
       Locale.setDefault(original);
   }
   ```
   
   3. Run the corresponding test or a small Java harness.
   
   Before the fix, the output is:
   
   ```text
   lower=ı upper=İ
   ```
   
   The same harness passes with the proposed fix and produces:
   
   ```text
   lower=i upper=I
   ```
   
   ### Expected behavior
   
   SQL case conversion should be deterministic and independent of the worker machine’s default locale. `lower("I")` should return `i` and `upper("i")` should return `I` regardless of the JVM locale.
   
   ### Environment
   
   - SeaTunnel branch: `dev`
   - Module: `seatunnel-transforms-v2`
   - JVM: Java 8+
   - Reproduction does not require an external connector, database, or cluster.
   
   I have prepared a focused fix on the fork branch `tagadearpit:fix/sql-string-case-locale`, replacing the implicit default locale with `Locale.ROOT` and adding a regression test. The focused `StringFunctionTest` passes with 34 tests, and the transforms module verification lifecycle passes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Transform-V2] SQL lower and upper depend on the worker default locale [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo closed issue #11949: [Transform-V2] SQL lower and upper depend on the worker default locale
URL: https://github.com/apache/seatunnel/issues/11949


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org