You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "viminizer (via GitHub)" <gi...@apache.org> on 2026/08/10 04:34:42 UTC
[PR] Fix PostgreSQL function projection column label case folding [shardingsphere]
viminizer opened a new pull request, #39393:
URL: https://github.com/apache/shardingsphere/pull/39393
Fixes #39392.
Changes proposed in this pull request:
- Fold unquoted PostgreSQL function names to lower case when deriving the column label of an unaliased function projection, matching PostgreSQL identifier folding.
- Strip quote characters from quoted function names and preserve their case, matching the guard `ProjectionIdentifierExtractEngine#getIdentifierValue` already applies to identifiers.
- Correct the PostgreSQL and openGauss test expectations that recorded the previous behavior, and cover the quoted case.
The function branch of `PostgreSQLProjectionIdentifierExtractor#getColumnNameFromExpression` returned the raw parsed name, while the aggregation branch below it routed through `getColumnNameFromFunction`, which lower cases. That is why `SUM` was already correct and `ABS` was not: `AggregationProjection#getColumnLabel` passes `AggregationType#name()`, which is always upper case, so that path had to fold to work at all, whereas the function branch receives the source text as typed.
The label is user visible. `QueryHeaderBuilderEngine#build` prefers `projection.getColumnLabel()` over the backend result set label.
### Verification
Built this branch, packaged ShardingSphere-Proxy, and ran it against PostgreSQL 16 with `t_order` sharded over `t_order_0` and `t_order_1`.
| Query | PostgreSQL | Proxy 5.5.3 | This branch |
|---|---|---|---|
| `SELECT ABS(user_id) FROM t_order` | `abs` | `ABS` | `abs` |
| `SELECT AbS(user_id) FROM t_order` | `abs` | `AbS` | `abs` |
| `SELECT MYABS(user_id) FROM t_order` | `myabs` | `MYABS` | `myabs` |
| `SELECT "MyFunc"(user_id) FROM t_order` | `MyFunc` | `"MyFunc"` | `MyFunc` |
| `SELECT abs(user_id) FROM t_order` | `abs` | `abs` | `abs` |
| `SELECT SUM(user_id) FROM t_order` | `sum` | `sum` | `sum` |
Checks that the change must not disturb, all unchanged on this branch:
| Query | Label |
|---|---|
| `SELECT user_id AS "MyAlias" FROM t_order` | `MyAlias` |
| `SELECT user_id AS my_alias FROM t_order` | `my_alias` |
| `SELECT user_id FROM t_order` | `user_id` |
| `SELECT PG_CATALOG.ABS(user_id) FROM t_order` | `abs` |
| `SELECT MAX(user_id), COUNT(*) FROM t_order` | `max`, `count` |
| `SELECT ABS(user_id) AS keep_me FROM t_order` | `keep_me` |
`infra/binder/dialect/postgresql` and `infra/binder/dialect/opengauss` pass with checkstyle and spotless clean.
### Notes
openGauss is included because `OpenGaussProjectionIdentifierExtractor` delegates every method to the PostgreSQL extractor and openGauss folds identifiers the same way. Its test carried the same expectation and is corrected alongside.
The `getColumnNameFromSubquery` TODO in the same class is a separate gap and is left untouched.
---
Before committing this PR, I'm sure that I have checked the following options:
- [x] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
- [x] I have self-reviewed the commit code.
- [ ] I have (or in comment I request) added corresponding labels for the pull request.
- [x] I have passed maven check locally : `./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e`.
- [ ] I have made corresponding changes to the documentation.
- [x] I have added corresponding unit tests for my changes.
- [x] I have updated the Release Notes of the current development version. For more details, see [Update Release Note](https://shardingsphere.apache.org/community/en/involved/contribute/contributor/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix PostgreSQL function projection column label case folding [shardingsphere]
Posted by "viminizer (via GitHub)" <gi...@apache.org>.
viminizer commented on PR #39393:
URL: https://github.com/apache/shardingsphere/pull/39393#issuecomment-5628467833
Three further defects turned up while building the Unicode-quoted path. They are outside the two blocking issues, so noting them here for the record.
### Out-of-range code points reached the binder as an internal error
`U&"\+FFFFFF"` parses, and `StringBuilder#appendCodePoint` then threw an unchecked `IllegalArgumentException` that surfaced as an internal error rather than a column label. Code points are now validated with `Character#isValidCodePoint` before being appended, and an invalid one falls back to emitting the escape character literally.
### The identifier path needed the same unquoting as the function path
`IdentifierValue` derives its quote character from the first character of the text, so an alias written as `U&"foo"` arrives with `QuoteCharacter.NONE` and its raw text intact. Folding it produced the label `u&"foo"` where PostgreSQL reports `foo`. `getIdentifierValue` now checks for the Unicode-quoted form too, not only `getColumnNameFromFunctionSegment`.
### The closing quote was located with `lastIndexOf`
Scanning backwards landed inside the `UESCAPE` string whenever that string contained a quote, which silently shifted the identifier boundary. The closing quote is now found by scanning forward from the end of the `U&"` prefix.
The doubled-quote collapse that accompanied it is dropped as dead code. The lexer rule at [`Literals.g4:24`](https://github.com/apache/shardingsphere/blob/master/parser/sql/engine/dialect/postgresql/src/main/antlr4/imports/postgresql/Literals.g4#L24) is `DQ_ ~'"'+ DQ_`, whose body excludes `"` entirely, so an embedded doubled quote cannot reach the binder.
### Coverage
`PostgreSQLIdentifierUtilsTest` covers the escaped-escape, non-hex, truncated and out-of-range branches. Both classes reach 100% branch coverage.
The `fold` Javadoc also notes that the ASCII-only rule assumes a multibyte server encoding; single-byte encodings downcase by server locale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix PostgreSQL function projection column label case folding [shardingsphere]
Posted by "viminizer (via GitHub)" <gi...@apache.org>.
viminizer commented on PR #39393:
URL: https://github.com/apache/shardingsphere/pull/39393#issuecomment-5627236261
Both blocking issues are addressed. Rebased onto `master` at 0d6b991878d, no conflicts.
The shared logic now lives in a new `PostgreSQLIdentifierUtils`, and `OpenGaussProjectionIdentifierExtractor` picks it up through its existing `PostgreSQLProjectionIdentifierExtractor` delegate.
### 1. Unicode-quoted function names
`isUnicodeQuoted` and `unquoteUnicode` are checked before any folding, in both `getIdentifierValue` and `getColumnNameFromFunctionSegment`. `SELECT U&"MyFunc"(id) FROM t_order` now yields the label `MyFunc`.
Decoding covers the default `\` escape, the `UESCAPE '<char>'` clause, the escaped escape character, the short `\XXXX` and long `\+XXXXXX` forms, and surrogate pairs. Malformed input, such as non-hex digits, a truncated escape, a trailing escape character, or an out-of-range code point, emits the escape character literally rather than throwing, since the parser has already accepted the token.
### 2. Locale-dependent folding
`fold` walks the char array and maps ASCII `A`-`Z` only, matching `downcase_identifier` in `scansup.c` for a multibyte encoding. No `toLowerCase` or `toUpperCase` call remains in `src/main` of either dialect.
`INITCAP` under a `tr-TR` default locale now folds to `initcap`, and `ÄBC` folds to `Äbc`.
### Coverage
Regression tests for both forms in both dialect paths, including the Turkish-locale and non-ASCII cases:
- `PostgreSQLIdentifierUtilsTest`: 17 tests
- `PostgreSQLProjectionIdentifierExtractorTest`: 14 tests
- `OpenGaussProjectionIdentifierExtractorTest`: 12 tests
All pass, and `-Pcheck checkstyle:check spotless:check` reports 0 violations on both modules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix PostgreSQL function projection column label case folding [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu merged PR #39393:
URL: https://github.com/apache/shardingsphere/pull/39393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix PostgreSQL function projection column label case folding [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu commented on PR #39393:
URL: https://github.com/apache/shardingsphere/pull/39393#issuecomment-5271052231
### Result
**Review Result: Not Mergeable**
Feedback Mode: Change Request
**Blocking Issues: 2**
The implementation still generates incorrect client-visible column labels for two valid PostgreSQL function-projection cases.
### Blocking Issues
1. Unicode-quoted function names are treated as unquoted text
- **Evidence:** [`getColumnNameFromFunctionSegment`](https://github.com/viminizer/shardingsphere/blob/09b4bf8d28265f57f1417290f1cf3e8206ab9e91/infra/binder/dialect/postgresql/src/main/java/org/apache/shardingsphere/infra/binder/postgresql/PostgreSQLProjectionIdentifierExtractor.java#L60-L64) recognizes quoting only when the first character is a quote. ShardingSphere’s PostgreSQL grammar accepts `U&"..."`, and a parser-to-binder reproduction produced `u&"myfunc"` for `U&"MyFunc"`. PostgreSQL defines this form as a quoted identifier whose case must be preserved and whose output name is `MyFunc`. [PostgreSQL 16 identifier rules](https://www.postgresql.org/docs/16/sql-syntax-lexical.html)
- **Impact:** A valid projection such as `SELECT U&"MyFunc"(id) FROM t_order` exposes an incorrect column label containing lowercased quoting syntax.
- **Required Change:** Handle PostgreSQL Unicode-quoted identifiers, including their escape semantics, before applying unquoted-name folding. Add focused PostgreSQL regression coverage for this form.
2. Unquoted function folding depends on JVM locale and Unicode rules
- **Evidence:** The newly routed path calls no-argument `String#toLowerCase()` through [`getColumnNameFromFunction`](https://github.com/viminizer/shardingsphere/blob/09b4bf8d28265f57f1417290f1cf3e8206ab9e91/infra/binder/dialect/postgresql/src/main/java/org/apache/shardingsphere/infra/binder/postgresql/PostgreSQLProjectionIdentifierExtractor.java#L39-L64). Under a Turkish locale, `INITCAP` produced code points `0131 006e 0131 0074 0063 0061 0070` instead of `initcap`; `ÄBC` became `äbc`. PostgreSQL 16 folds ASCII `A`–`Z` independently of locale and leaves multibyte identifier bytes unchanged. [PostgreSQL scanner source](https://github.com/postgres/postgres/blob/REL_16_STABLE/src/backend/parser/scansup.c#L42-L73)
- **Impact:** On affected JVM locales or with non-ASCII function names, Proxy/JDBC can return a column label different from PostgreSQL. openGauss inherits the same behavior through its PostgreSQL delegate.
- **Required Change:** Use deterministic PostgreSQL/openGauss-compatible identifier folding instead of JVM-default Unicode lowercasing. Cover Turkish-locale and non-ASCII cases for both dialect paths.
### Coverage
- Reviewed head: `09b4bf8d28265f57f1417290f1cf3e8206ab9e91`; merge base: `3c47902d3384ab21f573f38a45b01a175af1dd86`.
- Requirements reviewed: [issue #39392](https://github.com/apache/shardingsphere/issues/39392), PostgreSQL 16 identifier and output-column behavior, and openGauss delegation.
- All four authoritative changed files were accounted for: the PostgreSQL implementation, PostgreSQL and openGauss tests, and the release note.
- Behavior clusters reviewed: function-label normalization, quoted and unquoted boundaries, PostgreSQL/openGauss delegation, regression tests, and release visibility.
- Root-cause, blast-radius/contracts, tests/runtime/operations, and final convergence passes were completed. No unresolved gap could change the blocker set.
- Isolated PostgreSQL test: 8 tests passed. openGauss test with upstream dependency freshness (`-am`): 7 tests passed.
- Code-correctness review only; CI and GitHub Actions were not reviewed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org