You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zhangstar333 (via GitHub)" <gi...@apache.org> on 2024/04/23 12:12:00 UTC
[PR] [test]Ip arrow [doris]
zhangstar333 opened a new pull request, #34042:
URL: https://github.com/apache/doris/pull/34042
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079201893
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 merged PR #34042:
URL: https://github.com/apache/doris/pull/34042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2078983883
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095096623
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079021786
TeamCity be ut coverage result:
Function Coverage: 35.56% (8926/25103)
Line Coverage: 27.15% (73410/270391)
Region Coverage: 26.33% (37918/144015)
Branch Coverage: 23.11% (19305/83544)
Coverage Report: http://coverage.selectdb-in.cc/coverage/07f5cae477779b7bc6211de94c29b0a04ddd0832_07f5cae477779b7bc6211de94c29b0a04ddd0832/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095093379
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095443135
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095443193
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test]Ip arrow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2072140137
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test]Ip arrow [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2072150157
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #34042:
URL: https://github.com/apache/doris/pull/34042#discussion_r1590005743
##########
be/src/vec/data_types/serde/data_type_ipv4_serde.cpp:
##########
@@ -108,5 +109,29 @@ Status DataTypeIPv4SerDe::read_column_from_pb(IColumn& column, const PValues& ar
}
return Status::OK();
}
+
+void DataTypeIPv4SerDe::write_column_to_arrow(const IColumn& column, const NullMap* null_map,
+ arrow::ArrayBuilder* array_builder, int start,
+ int end, const cctz::time_zone& ctz) const {
+ const auto& col_data = static_cast<const ColumnIPv4&>(column).get_data();
Review Comment:
use assert_cast
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test]Ip arrow [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2078970821
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095112530
TeamCity be ut coverage result:
Function Coverage: 35.61% (8966/25178)
Line Coverage: 27.24% (73989/271575)
Region Coverage: 26.45% (38207/144427)
Branch Coverage: 23.21% (19456/83824)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b2198c4c74a138f25ccabd33ba4ea64243d7db0c_b2198c4c74a138f25ccabd33ba4ea64243d7db0c/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095277472
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #34042:
URL: https://github.com/apache/doris/pull/34042#discussion_r1590010904
##########
be/src/vec/data_types/serde/data_type_ipv6_serde.cpp:
##########
@@ -112,5 +114,88 @@ Status DataTypeIPv6SerDe::read_column_from_pb(IColumn& column, const PValues& ar
return Status::OK();
}
+void DataTypeIPv6SerDe::write_column_to_arrow(const IColumn& column, const NullMap* null_map,
+ arrow::ArrayBuilder* array_builder, int start,
+ int end, const cctz::time_zone& ctz) const {
+ const auto& col_data = static_cast<const ColumnIPv6&>(column).get_data();
+ auto& string_builder = assert_cast<arrow::StringBuilder&>(*array_builder);
+ for (size_t i = start; i < end; ++i) {
+ if (null_map && (*null_map)[i]) {
+ checkArrowStatus(string_builder.AppendNull(), column.get_name(),
+ array_builder->type()->name());
+ } else {
+ std::string ipv6_str = IPv6Value::to_string(col_data[i]);
+ checkArrowStatus(string_builder.Append(ipv6_str.c_str(), ipv6_str.size()),
+ column.get_name(), array_builder->type()->name());
+ }
+ }
+}
+
+void DataTypeIPv6SerDe::read_column_from_arrow(IColumn& column, const arrow::Array* arrow_array,
+ int start, int end,
+ const cctz::time_zone& ctz) const {
+ auto& col_data = static_cast<ColumnIPv6&>(column).get_data();
+ const auto* concrete_array = dynamic_cast<const arrow::StringArray*>(arrow_array);
Review Comment:
assert_cast?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095282905
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079014592
<details>
<summary>TPC-H: <b>Total hot run time: 40215 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 07f5cae477779b7bc6211de94c29b0a04ddd0832, data reload: false
------ Round 1 ----------------------------------
q1 17613 4388 4272 4272
q2 2023 191 189 189
q3 10503 1235 1209 1209
q4 11072 760 788 760
q5 7903 2793 2744 2744
q6 217 134 135 134
q7 1045 640 606 606
q8 9318 2143 2118 2118
q9 9027 6621 6578 6578
q10 8697 3735 3726 3726
q11 445 243 227 227
q12 389 213 223 213
q13 17757 2959 2937 2937
q14 274 226 222 222
q15 507 466 482 466
q16 496 388 376 376
q17 969 741 749 741
q18 8106 7442 7378 7378
q19 1621 1511 1492 1492
q20 682 315 297 297
q21 5014 3259 3977 3259
q22 342 271 277 271
Total cold run time: 114020 ms
Total hot run time: 40215 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4284 4233 4259 4233
q2 367 276 264 264
q3 2997 2753 2724 2724
q4 1879 1599 1599 1599
q5 5361 5300 5341 5300
q6 209 120 123 120
q7 2269 1916 1895 1895
q8 3219 3357 3363 3357
q9 8608 8549 8756 8549
q10 4033 3716 3663 3663
q11 584 484 494 484
q12 752 592 582 582
q13 17396 2960 2940 2940
q14 316 298 270 270
q15 522 475 489 475
q16 463 418 432 418
q17 1756 1466 1470 1466
q18 7617 7432 7454 7432
q19 1622 1538 1600 1538
q20 1936 1747 1785 1747
q21 4980 4815 4791 4791
q22 568 497 483 483
Total cold run time: 71738 ms
Total hot run time: 54330 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2078981391
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079267223
<details>
<summary>TPC-DS: <b>Total hot run time: 186229 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4196741439ac729eb27f3c189f20f600959f399, data reload: false
query1 906 368 349 349
query2 6380 2430 2330 2330
query3 6671 203 215 203
query4 23689 21765 21803 21765
query5 3919 439 423 423
query6 273 193 190 190
query7 4553 304 296 296
query8 253 203 195 195
query9 8296 2349 2353 2349
query10 414 260 271 260
query11 15340 14688 14821 14688
query12 115 93 87 87
query13 1644 360 359 359
query14 10605 6794 7866 6794
query15 258 170 165 165
query16 8113 255 247 247
query17 1713 546 572 546
query18 2100 279 266 266
query19 320 148 146 146
query20 86 85 80 80
query21 195 132 127 127
query22 4923 4822 4818 4818
query23 33950 33246 33312 33246
query24 10677 2933 2925 2925
query25 611 362 365 362
query26 1117 150 148 148
query27 2360 316 317 316
query28 7266 2001 1977 1977
query29 849 610 595 595
query30 238 148 159 148
query31 965 732 713 713
query32 90 52 50 50
query33 743 239 249 239
query34 1040 485 475 475
query35 810 668 663 663
query36 1089 939 897 897
query37 127 67 64 64
query38 3162 3004 2983 2983
query39 1600 1529 1548 1529
query40 195 126 125 125
query41 44 38 39 38
query42 105 97 98 97
query43 567 535 528 528
query44 1228 729 757 729
query45 265 268 263 263
query46 1068 741 710 710
query47 1957 1868 1899 1868
query48 357 287 292 287
query49 893 401 392 392
query50 794 385 378 378
query51 6759 6686 6694 6686
query52 104 87 94 87
query53 353 284 297 284
query54 305 228 233 228
query55 75 71 70 70
query56 234 224 214 214
query57 1203 1137 1109 1109
query58 211 196 198 196
query59 3428 3143 3125 3125
query60 240 228 259 228
query61 94 88 115 88
query62 656 453 453 453
query63 308 277 279 277
query64 8425 7146 7129 7129
query65 3095 3056 3043 3043
query66 831 324 333 324
query67 15251 15195 14930 14930
query68 5283 560 531 531
query69 494 301 319 301
query70 1182 1095 1174 1095
query71 419 266 276 266
query72 7262 2618 2459 2459
query73 699 317 317 317
query74 6511 6047 5997 5997
query75 3359 2623 2632 2623
query76 3142 938 984 938
query77 388 264 272 264
query78 10982 10129 10161 10129
query79 3639 510 522 510
query80 2202 497 437 437
query81 536 225 218 218
query82 815 98 99 98
query83 292 170 174 170
query84 271 89 86 86
query85 1951 264 265 264
query86 515 307 300 300
query87 3280 3100 3063 3063
query88 4822 2313 2315 2313
query89 489 380 384 380
query90 2021 182 181 181
query91 125 99 98 98
query92 61 50 47 47
query93 5057 504 500 500
query94 1157 181 180 180
query95 391 304 294 294
query96 607 267 257 257
query97 3131 2951 2971 2951
query98 236 218 212 212
query99 1221 876 851 851
Total cold run time: 289045 ms
Total hot run time: 186229 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095307354
<details>
<summary>TPC-H: <b>Total hot run time: 41905 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8a2e28492ab470902813b4ffefa01969908ec9e, data reload: false
------ Round 1 ----------------------------------
q1 17611 4371 4414 4371
q2 2007 190 192 190
q3 10464 1197 1111 1111
q4 10206 786 887 786
q5 7534 2687 2758 2687
q6 215 132 135 132
q7 1046 627 631 627
q8 9223 2145 2116 2116
q9 9866 6773 6826 6773
q10 9252 3983 3924 3924
q11 440 242 270 242
q12 476 235 226 226
q13 17196 3171 3242 3171
q14 262 222 232 222
q15 512 470 465 465
q16 476 395 394 394
q17 986 724 674 674
q18 8197 7810 7657 7657
q19 2644 1550 1482 1482
q20 627 314 316 314
q21 5267 4060 4103 4060
q22 359 281 283 281
Total cold run time: 114866 ms
Total hot run time: 41905 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4636 4376 4370 4370
q2 384 280 263 263
q3 3234 2948 2938 2938
q4 1865 1591 1593 1591
q5 5462 5453 5482 5453
q6 222 123 122 122
q7 2311 1985 1998 1985
q8 3283 3424 3394 3394
q9 8689 8774 8821 8774
q10 3989 3666 3823 3666
q11 584 504 508 504
q12 801 660 623 623
q13 16755 3140 3187 3140
q14 310 278 280 278
q15 549 488 475 475
q16 499 438 421 421
q17 1808 1483 1458 1458
q18 7682 7452 7309 7309
q19 1652 1555 1498 1498
q20 1981 1788 1737 1737
q21 10118 4962 4886 4886
q22 575 486 495 486
Total cold run time: 77389 ms
Total hot run time: 55371 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #34042:
URL: https://github.com/apache/doris/pull/34042#discussion_r1590010827
##########
be/src/vec/data_types/serde/data_type_ipv6_serde.cpp:
##########
@@ -112,5 +114,88 @@ Status DataTypeIPv6SerDe::read_column_from_pb(IColumn& column, const PValues& ar
return Status::OK();
}
+void DataTypeIPv6SerDe::write_column_to_arrow(const IColumn& column, const NullMap* null_map,
+ arrow::ArrayBuilder* array_builder, int start,
+ int end, const cctz::time_zone& ctz) const {
+ const auto& col_data = static_cast<const ColumnIPv6&>(column).get_data();
Review Comment:
assert_cast
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095113678
<details>
<summary>TPC-H: <b>Total hot run time: 41717 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b2198c4c74a138f25ccabd33ba4ea64243d7db0c, data reload: false
------ Round 1 ----------------------------------
q1 17853 4428 4235 4235
q2 2015 192 197 192
q3 10466 1125 1153 1125
q4 10188 735 793 735
q5 7478 2746 2686 2686
q6 230 134 135 134
q7 1063 623 626 623
q8 9231 2142 2091 2091
q9 9949 6766 6807 6766
q10 9402 3907 3916 3907
q11 440 235 229 229
q12 518 223 223 223
q13 17194 3165 3210 3165
q14 272 217 235 217
q15 525 470 479 470
q16 495 388 384 384
q17 971 649 678 649
q18 8265 7712 7643 7643
q19 1765 1574 1542 1542
q20 650 298 305 298
q21 5294 4157 4131 4131
q22 384 277 272 272
Total cold run time: 114648 ms
Total hot run time: 41717 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4528 4588 4401 4401
q2 374 268 257 257
q3 3151 2940 2959 2940
q4 1995 1611 1594 1594
q5 5378 5586 5482 5482
q6 207 126 125 125
q7 2365 1996 1989 1989
q8 3287 3397 3390 3390
q9 8662 8803 8847 8803
q10 4063 3708 3810 3708
q11 590 505 483 483
q12 779 619 632 619
q13 15802 3197 3220 3197
q14 301 270 275 270
q15 529 500 500 500
q16 500 436 441 436
q17 1783 1577 1527 1527
q18 7708 7623 7505 7505
q19 6881 1597 1550 1550
q20 1991 1759 1764 1759
q21 5083 4993 4976 4976
q22 571 492 493 492
Total cold run time: 76528 ms
Total hot run time: 56003 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079034006
<details>
<summary>TPC-DS: <b>Total hot run time: 186869 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 07f5cae477779b7bc6211de94c29b0a04ddd0832, data reload: false
query1 928 354 354 354
query2 6462 2301 2277 2277
query3 6665 207 206 206
query4 23852 21232 21219 21219
query5 4169 421 415 415
query6 276 190 168 168
query7 4594 297 281 281
query8 236 184 178 178
query9 8429 2321 2322 2321
query10 429 242 256 242
query11 14673 14115 14282 14115
query12 144 98 87 87
query13 1640 368 370 368
query14 10641 8258 8068 8068
query15 257 170 173 170
query16 8249 258 265 258
query17 1899 574 550 550
query18 2106 278 274 274
query19 343 153 152 152
query20 95 89 86 86
query21 198 127 130 127
query22 4994 4876 4816 4816
query23 34075 33296 33325 33296
query24 10937 2973 2959 2959
query25 637 387 388 387
query26 1392 153 148 148
query27 2872 317 309 309
query28 7478 2004 2003 2003
query29 916 612 602 602
query30 275 158 150 150
query31 999 739 710 710
query32 98 58 56 56
query33 758 257 248 248
query34 1025 493 470 470
query35 822 716 662 662
query36 1075 936 930 930
query37 138 67 70 67
query38 3131 3000 3022 3000
query39 1596 1528 1524 1524
query40 263 132 125 125
query41 45 42 43 42
query42 103 94 98 94
query43 570 556 560 556
query44 1234 719 734 719
query45 285 270 264 264
query46 1077 715 696 696
query47 1911 1843 1850 1843
query48 379 292 343 292
query49 1034 395 408 395
query50 783 372 389 372
query51 6845 6723 6788 6723
query52 103 86 88 86
query53 348 278 272 272
query54 295 230 231 230
query55 79 71 70 70
query56 234 220 217 217
query57 1213 1122 1145 1122
query58 227 197 194 194
query59 3499 3248 3173 3173
query60 262 236 233 233
query61 88 88 92 88
query62 634 442 446 442
query63 305 280 278 278
query64 8897 7153 7141 7141
query65 3081 3028 3031 3028
query66 1009 336 338 336
query67 15665 14989 15247 14989
query68 10014 550 551 550
query69 613 303 293 293
query70 1365 1057 1046 1046
query71 516 261 262 261
query72 8460 2679 2456 2456
query73 1614 322 313 313
query74 6546 6098 6150 6098
query75 4851 2708 2685 2685
query76 5939 1058 928 928
query77 633 266 262 262
query78 11220 10256 10315 10256
query79 10878 524 528 524
query80 1818 447 441 441
query81 499 224 220 220
query82 248 91 100 91
query83 215 167 167 167
query84 268 84 84 84
query85 998 272 267 267
query86 345 320 316 316
query87 3272 3074 3080 3074
query88 5116 2325 2321 2321
query89 525 387 376 376
query90 2004 183 185 183
query91 128 97 99 97
query92 64 49 48 48
query93 6615 527 505 505
query94 1367 185 184 184
query95 404 302 296 296
query96 641 261 263 261
query97 3111 2951 2932 2932
query98 244 217 218 217
query99 1190 844 844 844
Total cold run time: 311692 ms
Total hot run time: 186869 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079225721
TeamCity be ut coverage result:
Function Coverage: 35.53% (8927/25127)
Line Coverage: 27.14% (73424/270564)
Region Coverage: 26.31% (37920/144104)
Branch Coverage: 23.09% (19303/83582)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c4196741439ac729eb27f3c189f20f600959f399_c4196741439ac729eb27f3c189f20f600959f399/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2079208552
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095322107
<details>
<summary>TPC-DS: <b>Total hot run time: 185263 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8a2e28492ab470902813b4ffefa01969908ec9e, data reload: false
query1 907 353 345 345
query2 6419 2463 2374 2374
query3 6648 212 214 212
query4 23090 21334 21106 21106
query5 4110 413 417 413
query6 266 176 180 176
query7 4593 293 298 293
query8 232 187 188 187
query9 8454 2388 2372 2372
query10 427 243 246 243
query11 14770 14107 14217 14107
query12 134 89 90 89
query13 1630 364 377 364
query14 8943 6891 7598 6891
query15 210 173 174 173
query16 7742 261 267 261
query17 1769 573 563 563
query18 1953 280 280 280
query19 204 151 151 151
query20 91 85 86 85
query21 196 129 126 126
query22 5059 4833 4835 4833
query23 33702 33176 33156 33156
query24 6425 2912 2881 2881
query25 522 389 382 382
query26 695 156 186 156
query27 1833 310 312 310
query28 3670 2095 2055 2055
query29 834 590 605 590
query30 242 153 156 153
query31 993 747 731 731
query32 68 54 55 54
query33 522 246 239 239
query34 888 476 479 476
query35 754 730 667 667
query36 1028 904 877 877
query37 107 67 67 67
query38 3107 2991 3103 2991
query39 1592 1530 1567 1530
query40 199 139 126 126
query41 42 38 37 37
query42 106 97 93 93
query43 591 559 539 539
query44 1100 753 762 753
query45 274 260 228 228
query46 1069 742 719 719
query47 1930 1845 1829 1829
query48 378 300 287 287
query49 769 391 393 391
query50 774 384 393 384
query51 6796 6679 6673 6673
query52 102 93 90 90
query53 369 276 279 276
query54 259 236 232 232
query55 76 69 71 69
query56 241 216 225 216
query57 1197 1106 1110 1106
query58 211 191 200 191
query59 3250 3202 3255 3202
query60 244 232 229 229
query61 109 88 86 86
query62 593 441 452 441
query63 308 278 281 278
query64 7635 7274 7178 7178
query65 3074 3056 3054 3054
query66 791 350 326 326
query67 15314 14953 14889 14889
query68 6440 531 544 531
query69 546 310 307 307
query70 1158 1119 1153 1119
query71 482 264 265 264
query72 8108 2556 2295 2295
query73 731 316 313 313
query74 6403 6107 6048 6048
query75 3716 2597 2596 2596
query76 3774 1039 1044 1039
query77 577 268 270 268
query78 10929 10236 10302 10236
query79 8095 508 506 506
query80 1173 429 421 421
query81 479 218 225 218
query82 828 92 91 91
query83 199 160 164 160
query84 264 88 84 84
query85 964 266 262 262
query86 441 290 315 290
query87 3263 3060 3035 3035
query88 4935 2330 2331 2330
query89 558 376 381 376
query90 2181 194 186 186
query91 124 95 99 95
query92 57 46 46 46
query93 6814 536 506 506
query94 1075 181 179 179
query95 393 306 304 304
query96 622 263 260 260
query97 3135 2952 2970 2952
query98 225 215 218 215
query99 1244 865 850 850
Total cold run time: 282563 ms
Total hot run time: 185263 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095347106
TeamCity be ut coverage result:
Function Coverage: 35.62% (8969/25178)
Line Coverage: 27.25% (74018/271582)
Region Coverage: 26.47% (38232/144433)
Branch Coverage: 23.23% (19472/83828)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d8a2e28492ab470902813b4ffefa01969908ec9e_d8a2e28492ab470902813b4ffefa01969908ec9e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](serde) fix ipv4/v6 serde functions for arrow, orc, parquet format [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34042:
URL: https://github.com/apache/doris/pull/34042#issuecomment-2095120794
<details>
<summary>TPC-DS: <b>Total hot run time: 185767 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b2198c4c74a138f25ccabd33ba4ea64243d7db0c, data reload: false
query1 914 368 347 347
query2 6442 2370 2491 2370
query3 6643 208 210 208
query4 23290 21733 21923 21733
query5 4210 507 458 458
query6 1021 204 200 200
query7 5836 288 290 288
query8 521 203 198 198
query9 11268 2395 2404 2395
query10 1136 264 268 264
query11 15382 14227 14144 14144
query12 140 90 85 85
query13 1639 361 380 361
query14 9886 6675 8282 6675
query15 258 164 178 164
query16 8112 249 257 249
query17 2265 556 551 551
query18 2102 274 270 270
query19 335 165 150 150
query20 94 86 82 82
query21 193 132 128 128
query22 5080 4860 4841 4841
query23 34428 33451 33313 33313
query24 10177 2997 2831 2831
query25 595 364 361 361
query26 1149 158 148 148
query27 2279 318 330 318
query28 6887 2018 1999 1999
query29 867 605 597 597
query30 246 153 155 153
query31 965 765 711 711
query32 91 52 59 52
query33 736 268 239 239
query34 998 480 481 480
query35 814 663 658 658
query36 1089 890 936 890
query37 134 70 64 64
query38 3227 2988 3042 2988
query39 1572 1529 1565 1529
query40 206 131 126 126
query41 41 40 39 39
query42 108 102 98 98
query43 567 535 551 535
query44 1226 725 748 725
query45 265 242 243 242
query46 1072 723 698 698
query47 1913 1863 1835 1835
query48 390 293 301 293
query49 850 395 405 395
query50 778 409 385 385
query51 6704 6550 6650 6550
query52 100 95 108 95
query53 353 286 285 285
query54 329 242 247 242
query55 82 73 73 73
query56 251 233 216 216
query57 1210 1146 1160 1146
query58 215 198 198 198
query59 3399 3158 3250 3158
query60 258 231 233 231
query61 91 91 90 90
query62 647 444 437 437
query63 313 291 285 285
query64 8580 7202 7270 7202
query65 3137 3045 3045 3045
query66 795 336 344 336
query67 15586 15038 14986 14986
query68 5265 524 528 524
query69 480 307 302 302
query70 1202 1119 1091 1091
query71 417 271 267 267
query72 7939 2615 2373 2373
query73 702 326 323 323
query74 6503 6039 6053 6039
query75 3336 2634 2602 2602
query76 3348 1051 1011 1011
query77 418 268 262 262
query78 10951 10447 10137 10137
query79 3354 510 495 495
query80 2005 431 426 426
query81 561 228 220 220
query82 800 102 97 97
query83 277 174 170 170
query84 265 88 86 86
query85 2148 270 271 270
query86 500 315 275 275
query87 3320 3094 3068 3068
query88 4740 2329 2325 2325
query89 499 389 382 382
query90 2099 181 179 179
query91 125 99 96 96
query92 60 46 45 45
query93 4688 506 498 498
query94 1233 178 177 177
query95 383 295 287 287
query96 614 275 257 257
query97 3162 2951 2951 2951
query98 243 221 222 221
query99 1211 873 883 873
Total cold run time: 295335 ms
Total hot run time: 185767 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org