You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2026/10/16 05:12:04 UTC

[PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Gabriel39 opened a new pull request, #69171:
URL: https://github.com/apache/doris/pull/69171

   ## Problem and behavior
   
   `branch-4.2` is missing several external type mapping fixes already merged into master. This adapted backport aligns these paths while retaining the branch's existing VARBINARY and TIMESTAMPTZ support:
   
   | Path | Result | Upstream |
   | --- | --- | --- |
   | Parquet/ORC file TVFs | Binary fields infer VARBINARY with the legacy option absent, false, or true, including nested fields | #68786 |
   | Oracle JDBC | BINARY_FLOAT maps to FLOAT; BINARY_DOUBLE maps to DOUBLE | #68200 |
   | SQL Server JDBC | Resolve user-defined aliases by JDBC type code; strip IDENTITY decoration only when the base type agrees with that code | #67916 |
   | Iceberg metadata scanner | Preserve signed epoch microseconds and timestamp-without-zone wall-clock values through nested structs, arrays, and maps | #68985 |
   
   **Merge after #69163**, which supplies the V2 binary/UUID reader changes and nanosecond timestamp support. This PR does not duplicate those BE changes. The Iceberg scanner keeps this branch's full-schema source indexes rather than copying master's projected-row indexing. Native UUID and the master JDBC scanner architecture are outside this backport.
   
   Binary TVF columns now display hexadecimal values. Direct CTAS into an internal VARBINARY column is rejected, consistently with master; explicit HEX conversion can be used to materialize bytes into an internal string column. Regression coverage checks this boundary and byte preservation for both file readers and all legacy option states. Existing TVF goldens are updated from execution against the combined behavior, including timestamp semantics from #69163; unrelated numeric formatting is preserved.
   
   ## Validation
   
   - Targeted Java tests: 15 passed, with failures reproduced before the fix for all four changed paths. Changed production sources and tests were compiled against branch-compatible build dependencies.
   - FE Checkstyle: passed for the FE core and Iceberg metadata scanner reactor.
   - Local regression comparison: 14 suites passed (12 HDFS Parquet/ORC TVF suites, HTTP TVF, and the new binary mapping suite); no skipped suites.
   - Local file regressions use the prebuilt #69163 runtime with the changed FE classes. A full FE/BE rebuild and live Oracle/SQL Server/Iceberg integration were not run. Iceberg timestamp unit coverage and the upstream metadata regression are included.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69171:
URL: https://github.com/apache/doris/pull/69171#discussion_r4292655737


##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java:
##########
@@ -253,12 +253,9 @@ && getTFileType() != TFileType.FILE_S3) {
             throw new AnalysisException("Lance format is supported only by local() and s3() TVFs");
         }
 
-        // The catalog property was removed, but this TVF-only option must remain explicit because
-        // changing an ad-hoc TVF result schema also breaks CTAS type inference.
-        String enableMappingVarbinaryStr = getOrDefaultAndRemove(copiedProps,
-                FileFormatConstants.PROP_ENABLE_MAPPING_VARBINARY, "false");
-        fileFormatProperties.enableMappingVarbinary = Boolean.parseBoolean(enableMappingVarbinaryStr);
-
+        // Binary file fields must preserve bytes even when a legacy opt-out is supplied.
+        copiedProps.remove(FileFormatConstants.PROP_ENABLE_MAPPING_VARBINARY);
+        fileFormatProperties.enableMappingVarbinary = true;

Review Comment:
   [P2] Keep Parquet ENUM columns textual with the new binary default
   
   This also changes ENUM-annotated BYTE_ARRAY columns from STRING to VARBINARY. Schema discovery always uses the V1 ParquetReader, whose logical/converted-type switches omit ENUM and fall back to BYTE_ARRAY; the changed group6/byte_array.parquet golden already shows enum_smith becoming hex. [Parquet defines ENUM as UTF-8 text for applications without a native enum type](https://parquet.apache.org/docs/file-format/types/logicaltypes/#enum). As a result, CTAS selecting only an ENUM field now fails the VARBINARY create-table check, even when enable_mapping_varbinary=false is supplied. Handle both modern and legacy ENUM annotations as STRING during schema inference, and keep the ENUM golden expectations textual.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6260687252

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6248436383

   Fixed the remaining UUID/BINARY review finding and the external-regression failure:
   
   - Explicit Hive BINARY targets retain the original 16 bytes in the legacy Parquet and ORC readers. File Scanner V2 now converts native UUID values back to big-endian file bytes, including nested values.
   - Added Hive coverage for Parquet/ORC, both scanner settings, strict/non-strict casts, scalar/array/map/struct values, NULLs, empty collections, byte length and filtering. A direct legacy Parquet reader test also covers plain/dictionary fixtures across row groups and 37-row batches, since timestamp-semantics negotiation can force V2 independently of the session flag.
   - Regenerated the `test_hdfs_parquet_group2` golden output for ENUM text. The generated output also normalizes three equivalent floating-point representations.
   
   Validation: all four focused C++ tests passed after confirming the old-code failures; full ASAN BE build (including glibc compatibility), formatting/diff checks, and four regression suites passed: `test_hive_uuid_binary`, `test_hdfs_parquet_group2`, `test_uuid_parquet_native`, `test_uuid_orc_varbinary_mapping`.
   
   The previous FE coverage job 1072210 failed while downloading Maven from the internal OSS URL, before tests started. The external-regression failure in job 1072171 was the ENUM golden mismatch fixed here; its other four failures were already muted (S3 timeouts / unsupported Paimon OBS filesystem).
   
   Pushed commit `32fd2fedcdb` and requested fresh `run buildall` and `/review` checks. Those new checks are pending.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on code in PR #69171:
URL: https://github.com/apache/doris/pull/69171#discussion_r4296114767


##########
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcOracleClient.java:
##########
@@ -208,6 +208,10 @@ protected Type jdbcTypeToDoris(JdbcFieldSchema fieldSchema) {
                     precision = scale;
                 }
                 return createDecimalOrStringType(precision, scale);
+            // Oracle binary floating-point types retain IEEE precision instead of becoming unsupported.
+            case "BINARY_FLOAT":
+                return Type.FLOAT;
+            case "BINARY_DOUBLE":

Review Comment:
   Fixed by setting Oracle’s SetFloatAndDoubleUseBinary connection property before Hikari opens connections. This keeps FLOAT/DOUBLE inserts on the binary IEEE binding path rather than converting them through NUMBER. Added driver-contract/unit coverage and an INSERT/readback regression for large and small doubles, signed floats, and NULLs. Verified against a real Oracle database: the previous executor fails on setDouble(1e200) with Overflow; the fixed executor passes all five Java tests and the Doris JDBC regression block.



##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/ExternalFileTableValuedFunction.java:
##########
@@ -253,12 +253,9 @@ && getTFileType() != TFileType.FILE_S3) {
             throw new AnalysisException("Lance format is supported only by local() and s3() TVFs");
         }
 
-        // The catalog property was removed, but this TVF-only option must remain explicit because
-        // changing an ad-hoc TVF result schema also breaks CTAS type inference.
-        String enableMappingVarbinaryStr = getOrDefaultAndRemove(copiedProps,
-                FileFormatConstants.PROP_ENABLE_MAPPING_VARBINARY, "false");
-        fileFormatProperties.enableMappingVarbinary = Boolean.parseBoolean(enableMappingVarbinaryStr);
-
+        // Binary file fields must preserve bytes even when a legacy opt-out is supplied.
+        copiedProps.remove(FileFormatConstants.PROP_ENABLE_MAPPING_VARBINARY);
+        fileFormatProperties.enableMappingVarbinary = true;

Review Comment:
   Fixed shared Parquet schema inference for both modern ENUM logical annotations and legacy ConvertedType::ENUM: both remain STRING independently of the raw-binary option. Added a schema unit matrix and an ENUM-only CTAS/readback regression with the option set to both false and true. The focused schema test fails against the old implementation and passes with the fix; the HDFS regression suite also passes. Logical UUID annotations now take precedence over the raw-binary default as well, addressing the related CI regressions without changing the V1 decoder chain.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6218434203

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6211184755

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6251384275

   # BE Regression && UT Coverage Report
   Increment line coverage `88.89% (32/36)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69171_32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6_merge/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69171_32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6_merge/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 76.31% (34328/44983) |
   | Line Coverage     | 61.05% (386834/633668) |
   | Region Coverage   | 57.90% (326043/563086) |
   | Branch Coverage   | 58.82% (150038/255101) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6248433540

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6208589654

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6211183920

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6218434810

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6208585702

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6211117889

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6222659636

   # BE Regression && UT Coverage Report
   Increment line coverage `100.00% (13/13)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69171_e18aff1aae086842946cc98237cb89dc45698b8f_merge/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69171_e18aff1aae086842946cc98237cb89dc45698b8f_merge/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 76.18% (34302/45026) |
   | Line Coverage     | 61.10% (389039/636708) |
   | Region Coverage   | 57.83% (327610/566514) |
   | Branch Coverage   | 58.78% (150766/256501) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6251328355

   # BE UT Coverage Report
   Increment line coverage `88.89% (32/36)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6_32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6_32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 63.99% (29639/46318) |
   | Line Coverage     | 48.81% (312000/639191) |
   | Region Coverage   | 44.64% (250981/562187) |
   | Branch Coverage   | 46.26% (118175/255433) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #69171:
URL: https://github.com/apache/doris/pull/69171


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6208587545

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6221446340

   # FE UT Coverage Report
   Increment line coverage `  89.47% (34/38)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69171_e18aff1aae086842946cc98237cb89dc45698b8f/fe_increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69171_e18aff1aae086842946cc98237cb89dc45698b8f/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6251465222

   # FE Regression Coverage Report
   Increment line coverage `  1.58% (3/190)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69171_32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69171_32fd2fedcdbfe7a51ab8df129f12b3f71638a2a6_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6211118490

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69171:
URL: https://github.com/apache/doris/pull/69171#discussion_r4296299676


##########
be/src/format/parquet/schema_desc.cpp:
##########
@@ -316,12 +317,8 @@ std::pair<DataTypePtr, bool> FieldDescriptor::convert_to_doris_type(
     } else if (logicalType.__isset.JSON) {
         ans.first = DataTypeFactory::instance().create_data_type(TYPE_STRING, nullable);
     } else if (logicalType.__isset.UUID) {
-        if (_enable_mapping_varbinary) {
-            ans.first = DataTypeFactory::instance().create_data_type(TYPE_VARBINARY, nullable, -1,
-                                                                     -1, 16);
-        } else {
-            ans.first = DataTypeFactory::instance().create_data_type(TYPE_STRING, nullable);
-        }
+        // The raw-binary default must not override UUID's existing canonical-text contract.
+        ans.first = DataTypeFactory::instance().create_data_type(TYPE_STRING, nullable);

Review Comment:
   [P1] Preserve raw UUID bytes for schema-declared BINARY columns
   
   This shared parser also serves Hive `BINARY` columns, which map to `VARBINARY` with binary mapping enabled. Registering the existing `uuid_plain.parquet` fixture as a Hive table with `u BINARY` makes id=1's sixteen zero bytes take `UUIDStringConverter` and then `VarBinaryConverter<STRING,VARBINARY>`, so `LENGTH(u)` changes from 16 to 36 and `HEX(u)` contains the canonical UUID's ASCII bytes. Only Iceberg enables `preserve_binary_uuid`. The companion V2 change similarly emits `UUID`, but the mapper's UUID-to-VARBINARY cast is rejected by `create_varbinary_wrapper`, which accepts only STRING; annotated ORC binary targets have the same gap. Preserve a raw carrier for explicitly declared binary targets or add a bytes-preserving conversion, and cover Hive BINARY plus nested children with both scanner versions. This is distinct from the fixed ENUM TVF/CTAS issue.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6220668008

   # BE UT Coverage Report
   Increment line coverage `100.00% (13/13)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/e18aff1aae086842946cc98237cb89dc45698b8f_e18aff1aae086842946cc98237cb89dc45698b8f/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/e18aff1aae086842946cc98237cb89dc45698b8f_e18aff1aae086842946cc98237cb89dc45698b8f/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 63.88% (29653/46418) |
   | Line Coverage     | 48.82% (313683/642469) |
   | Region Coverage   | 44.56% (252178/565911) |
   | Branch Coverage   | 46.13% (118527/256953) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69171:
URL: https://github.com/apache/doris/pull/69171#discussion_r4291341811


##########
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcOracleClient.java:
##########
@@ -208,6 +208,10 @@ protected Type jdbcTypeToDoris(JdbcFieldSchema fieldSchema) {
                     precision = scale;
                 }
                 return createDecimalOrStringType(precision, scale);
+            // Oracle binary floating-point types retain IEEE precision instead of becoming unsupported.
+            case "BINARY_FLOAT":
+                return Type.FLOAT;
+            case "BINARY_DOUBLE":

Review Comment:
   [P2] Use Oracle binary bindings for the newly supported floats
   
   These mappings also enable JDBC INSERT targets, but `BaseJdbcExecutor.insertColumn` still uses `setFloat`/`setDouble`. [Oracle's JDBC contract](https://docs.oracle.com/en/database/oracle/oracle-database/19/jjdbc/Oracle-extensions.html#GUID-FC9510C6-8FF2-41A7-864A-890B85A316BC) says those setters encode values as NUMBER unless `SetFloatAndDoubleUseBinary` is enabled; `OracleJdbcExecutor` does not enable that option or use native binary setters. For example, copying a `BINARY_DOUBLE` source row containing `1e200` into another `BINARY_DOUBLE` column reaches `setDouble` and exceeds [NUMBER's range](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/Data-Types.html#GUID-75209AF6-476D-4C44-A5DC-5FA70D701B78), although the value is valid for both columns. Configure binary binding or use Oracle's binary setters when enabling these types, and cover this with an INSERT/round-trip test.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6260686679

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on code in PR #69171:
URL: https://github.com/apache/doris/pull/69171#discussion_r4306569960


##########
be/src/format/parquet/schema_desc.cpp:
##########
@@ -316,12 +317,8 @@ std::pair<DataTypePtr, bool> FieldDescriptor::convert_to_doris_type(
     } else if (logicalType.__isset.JSON) {
         ans.first = DataTypeFactory::instance().create_data_type(TYPE_STRING, nullable);
     } else if (logicalType.__isset.UUID) {
-        if (_enable_mapping_varbinary) {
-            ans.first = DataTypeFactory::instance().create_data_type(TYPE_VARBINARY, nullable, -1,
-                                                                     -1, 16);
-        } else {
-            ans.first = DataTypeFactory::instance().create_data_type(TYPE_STRING, nullable);
-        }
+        // The raw-binary default must not override UUID's existing canonical-text contract.
+        ans.first = DataTypeFactory::instance().create_data_type(TYPE_STRING, nullable);

Review Comment:
   Fixed in 32fd2fedcdb. Legacy Parquet now retains its raw carrier for an explicit VARBINARY destination; legacy ORC decodes the annotated binary carrier directly for BINARY targets; V2 converts native UUID values to the original big-endian 16 bytes. These paths also preserve nested values and NULLs.
   
   Added scalar/array cast tests, direct legacy Parquet reader coverage with plain/dictionary fixtures over 37-row batches and multiple row groups, and a Hive Parquet/ORC regression covering arrays/maps/structs, empty collections, NULLs, strict modes and scanner settings. The legacy-reader test is explicit because timestamp-semantics negotiation can force Parquet V2 despite the session flag.
   
   Validated the old-code failures (36-byte text instead of 16 bytes, unsupported UUID-to-VARBINARY cast) and the fixed results: all four focused C++ tests and four regression suites passed on an ASAN BE build. Existing native UUID TVF behavior remains covered.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](external) Backport missing type mapping behavior to branch-4.2 [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69171:
URL: https://github.com/apache/doris/pull/69171#issuecomment-6248433260

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org