You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "Tlinian (via GitHub)" <gi...@apache.org> on 2026/07/10 08:25:46 UTC

[PR] [Improve][Connector-Jdbc] Add YashanDB vector type support [seatunnel]

Tlinian opened a new pull request, #11397:
URL: https://github.com/apache/seatunnel/pull/11397

   - Upgrade YashanDB JDBC driver from 1.9.24 to 1.10.6 for com.yashandb.vector.Vector API
   - Split SUPPORT_VECTOR into SUPPORT_FLOAT32_VECTOR and SUPPORT_FLOAT64_VECTOR for proper float32/float64 vector distinction in ARRAY-to-VECTOR reconversion
   - Fix vector read path: add instanceof Vector guard with String fallback to prevent ClassCastException when JDBC driver returns non-Vector objects
   - Fix vector write path: separate ARRAY and VECTOR handling, use Vector.ofFloat32Values() with primitive float[] instead of unsafe (Object[]) cast and (ArrayType) cast on VectorType
   - Fix setArray: correct duplicate SUPPORT_FLOAT32_VECTOR condition to SUPPORT_FLOAT64_VECTOR, convert Object[] to primitive float[]/double[] before passing to Vector factory methods
   - Fix ARRAY null handling: add missing return after setNull to prevent NPE in setArray
   
   <!--
   
   Thank you for contributing to SeaTunnel! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   ## Contribution Checklist
     - Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/seatunnel/issues).
     - Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
     - Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
   -->
   
   ### Purpose of this pull request
   
   <!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released SeaTunnel versions or within the unreleased branches such as dev.
   If no, write 'No'.
   If you are adding/modifying connector documents, please follow our new specifications: https://github.com/apache/seatunnel/issues/4544.
   -->
   
   
   ### How was this patch tested?
   
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   If you are adding E2E test cases, maybe refer to https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/resources/mysqlcdc_to_mysql.conf, here is a good example.
   -->
   
   
   ### Check list
   
   * [ ] If any new Jar binary package adding in your PR, please add License Notice according
     [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
   * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR.
   * [ ] If you are contributing the connector code, please check that the following files are updated:
     1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
     2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
     3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
     4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
     5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][Connector-V2] Improve YashanDB vector type conversion [seatunnel]

Posted by "Tlinian (via GitHub)" <gi...@apache.org>.
Tlinian commented on PR #11397:
URL: https://github.com/apache/seatunnel/pull/11397#issuecomment-4964530492

   > @Tlinian Thank you very much for supplementing the data types supported by YashanDB in the future. I think we need to add an E2E that can cover all the types currently supported. Thank you again for your contribution🚀
   
   @nzw921rx Thanks for your suggestion!
   
   Currently, the vector-supported version of YashanDB has not been officially released to the public, which means we cannot upgrade the database version in the CI environment to run and verify these vector E2E tests at the moment.
   To move forward, how would you prefer we handle this?
   
   - We can write the E2E test cases now but mark them to be skipped in CI for the time being.
   
   - We can add unit tests with mocked data first to cover the logic, and implement the E2E tests later once the vector version is officially released and available in CI.
   
   Let me know what you think! Thank you again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][Connector-Jdbc] Add YashanDB vector type support [seatunnel]

Posted by "Tlinian (via GitHub)" <gi...@apache.org>.
Tlinian commented on PR #11397:
URL: https://github.com/apache/seatunnel/pull/11397#issuecomment-4933536051

   @DanielLeens @OctoberWithYou Hi! I have submitted this PR to add YashanDB vector type support. Could you please help review it when you are free? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][Connector-V2] Improve YashanDB vector type conversion [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11397:
URL: https://github.com/apache/seatunnel/pull/11397


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][Connector-V2] Improve YashanDB vector type conversion [seatunnel]

Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11397:
URL: https://github.com/apache/seatunnel/pull/11397#issuecomment-4968399593

   Thanks for the detailed context.
   
   Given the current constraint, I would prefer the second option: add focused unit coverage for the vector conversion path now, and leave the E2E test for a follow-up once the vector-enabled YashanDB build is publicly available and runnable in CI. A skipped E2E would not really give us merge evidence on the current branch.
   
   When you update the PR, please make the unit tests cover the concrete vector types this branch claims to support, and mention clearly in the PR that the end-to-end validation depends on the unreleased database version. Once that update is pushed, I'm happy to re-review.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][Connector-V2] Improve YashanDB vector type conversion [seatunnel]

Posted by "Tlinian (via GitHub)" <gi...@apache.org>.
Tlinian commented on PR #11397:
URL: https://github.com/apache/seatunnel/pull/11397#issuecomment-4991281514

   @DanielLeens @nzw921rx Hi, I have updated the PR according to your suggestions:
   
   - Added focused unit tests covering the concrete vector types (FLOAT32 and FLOAT64) that this branch supports.
   
   - Updated the PR description to clearly state that the E2E validation depends on the unreleased database version.
   
   Could you please take another look when you are free? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org