You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@wayang.apache.org by "AliRana30 (via GitHub)" <gi...@apache.org> on 2026/10/10 23:32:18 UTC
[PR] Validate and escape table identifiers in JDBC sink operators [wayang]
AliRana30 opened a new pull request, #839:
URL: https://github.com/apache/wayang/pull/839
### Key Changes
- **SqlIdentifierUtils**: Added a utility in `wayang-basic` to strictly validate single-part and multipart table/column identifiers using regex patterns (`^[a-zA-Z_][a-zA-Z0-9_$]*(?:\.[a-zA-Z_][a-zA-Z0-9_$]*)*$`) and escape/quote identifiers according to database dialect.
- **TableSink**: Integrated validation in the constructor and setter so any invalid or malicious table/column identifier is rejected early with an `IllegalArgumentException`.
- **JdbcTableSinkOperator & JdbcExecutor**: Quoted table names in generated `DROP TABLE IF EXISTS`, `CREATE TABLE ... AS`, and `INSERT INTO` clauses. Multipart identifiers (such as Trino `catalog.schema.table`) are quoted segment-by-segment (`"catalog"."schema"."table"`).
- **JavaTableSink & SparkTableSink**: Updated to use `SqlIdentifierUtils` for safe identifier quoting.
- **Unit Tests**: Added `SqlIdentifierUtilsTest` and updated `JdbcTableSinkExecutorTest` to assert proper quoting and rejection of SQL injection payloads.
## Fixes : #836.
### Verification
- `SqlIdentifierUtilsTest` (10 tests passed)
- `JdbcTableSinkExecutorTest` (8 tests passed)
- `JavaTableSinkTest` (7 tests passed)
- `SparkTableSinkTest` (10 tests passed)
- `apache-rat:check` (0 unapproved licenses)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "AliRana30 (via GitHub)" <gi...@apache.org>.
AliRana30 commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6401111407
@novatechflow Thanks for the review and catching the locale issue!
- Updated all identifier case folding in `SqlIdentifierUtils` and JDBC URL detection in `SqlTypeUtils` to use `Locale.ROOT`.
- Added Turkish locale (`tr-TR`) regression tests in `SqlIdentifierUtilsTest`, `SqlTypeUtilsTest`, and `JavaTableSinkTest` (verifying live HSQLDB table operations under Turkish locale).
- Resolved the CI failures in `Backend / Build` and `Docker / Build` caused by dummy pattern templates in `TableSinkMapping` (`new TableSink<Record>(null, null, "", new String[0])`) crashing during pattern registration; validation is strictly enforced during sink operator creation and query generation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "novatechflow (via GitHub)" <gi...@apache.org>.
novatechflow commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6261890746
Thanks for the contribution. Pls always add regression tests.
- Quoting changes identifier case handling. On HSQLDB, an existing unquoted `target_append` becomes `TARGET_APPEND`, but the new SQL targets `"target_append"`. Append fails, and overwrite leaves the original table untouched.
- The shared validation rejects valid BigQuery names such as `my-project.sales.orders`, including the documented backtick-quoted form.
Reproduced both with focused Java 17 checks. Preserve identifier lookup semantics & handle dialect-specific names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "AliRana30 (via GitHub)" <gi...@apache.org>.
AliRana30 commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6390356595
Thanks for the review @novatechflow . Addressed both issues and added regression tests:
- **Case-folding semantics:** Preserved standard identifier lookup via `DatabaseMetaData` case folding (folding unquoted identifiers to uppercase on HSQLDB/H2/Oracle/Derby and lowercase on Postgres, while preserving already-quoted names as-is). Unquoted tables like `target_append` now properly match catalog tables on append and overwrite.
- **BigQuery names:** Added support for hyphenated identifiers (`my-project.sales.orders`) and documented backtick forms (whole-quoted and per-segment).
- **Regression tests:** Added live HSQLDB tests covering append, overwrite, and quoted case-preservation semantics, as well as BigQuery validation test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "novatechflow (via GitHub)" <gi...@apache.org>.
novatechflow commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6335688098
Two small things, reproduced on Java 17:
- Mixed quoting breaks: `"PUBLIC".target_append` becomes `"""PUBLIC"""."TARGET_APPEND"`, and causes lookup failures.
- Case folding depends on system locale: under Turkish locale, input_table becomes `"İNPUT_TABLE"` and fails. Use `Locale.ROOT`, that should avoid that.
I think the quoting utility needs per-segment quote handling and locale-independent folding before approval. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "AliRana30 (via GitHub)" <gi...@apache.org>.
AliRana30 commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6328150195
@novatechflow I have addressed both issues and added regression tests covering both case folding and BigQuery formats. Verified on Java 17
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "novatechflow (via GitHub)" <gi...@apache.org>.
novatechflow merged PR #839:
URL: https://github.com/apache/wayang/pull/839
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "AliRana30 (via GitHub)" <gi...@apache.org>.
AliRana30 commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6103349373
@zkaoudi PR is up for your review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Validate and escape table identifiers in JDBC sink operators [wayang]
Posted by "novatechflow (via GitHub)" <gi...@apache.org>.
novatechflow commented on PR #839:
URL: https://github.com/apache/wayang/pull/839#issuecomment-6396093917
Ran a short test together with the other review, also great progress. Mixed quoting is fixed; the locale bug remains:
the code still uses parameterless `toUpperCase()` / `toLowerCase()`. Both need `Locale.ROOT`, plus a Turkish-locale regression test. My locale test found only Turkish, not sure if we also hit others. So `locale.ROOT` seems the best solution. We can't assume that users set their console to English.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org