You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/08 20:30:27 UTC
[GitHub] [arrow] bkietz opened a new pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
bkietz opened a new pull request #8403:
URL: https://github.com/apache/arrow/pull/8403
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #8403:
URL: https://github.com/apache/arrow/pull/8403#discussion_r502047888
##########
File path: cpp/src/parquet/arrow/arrow_reader_writer_test.cc
##########
@@ -2689,6 +2689,26 @@ TEST(TestArrowReadWrite, DictionaryColumnChunkedWrite) {
::arrow::AssertTablesEqual(*expected_table, *result, false);
}
+TEST(TestArrowReadWrite, NonUniqueDictionaryValues) {
+ // ARROW-10237
+ auto dict_with_dupes = ArrayFromJSON(::arrow::utf8(), R"(["a", "a", "b"])");
+ auto indices = ArrayFromJSON(::arrow::int32(), R"([0, 1, 0, 1])");
Review comment:
Okay, I'll try all permutations of 0,1,2,null
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #8403:
URL: https://github.com/apache/arrow/pull/8403#discussion_r502047888
##########
File path: cpp/src/parquet/arrow/arrow_reader_writer_test.cc
##########
@@ -2689,6 +2689,26 @@ TEST(TestArrowReadWrite, DictionaryColumnChunkedWrite) {
::arrow::AssertTablesEqual(*expected_table, *result, false);
}
+TEST(TestArrowReadWrite, NonUniqueDictionaryValues) {
+ // ARROW-10237
+ auto dict_with_dupes = ArrayFromJSON(::arrow::utf8(), R"(["a", "a", "b"])");
+ auto indices = ArrayFromJSON(::arrow::int32(), R"([0, 1, 0, 1])");
Review comment:
Okay, I'll try all permutations of 0,1,2,null
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz closed pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
bkietz closed pull request #8403:
URL: https://github.com/apache/arrow/pull/8403
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #8403:
URL: https://github.com/apache/arrow/pull/8403#discussion_r502044409
##########
File path: cpp/src/parquet/arrow/arrow_reader_writer_test.cc
##########
@@ -2689,6 +2689,26 @@ TEST(TestArrowReadWrite, DictionaryColumnChunkedWrite) {
::arrow::AssertTablesEqual(*expected_table, *result, false);
}
+TEST(TestArrowReadWrite, NonUniqueDictionaryValues) {
+ // ARROW-10237
+ auto dict_with_dupes = ArrayFromJSON(::arrow::utf8(), R"(["a", "a", "b"])");
+ auto indices = ArrayFromJSON(::arrow::int32(), R"([0, 1, 0, 1])");
Review comment:
You might want to check all possible index values.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #8403:
URL: https://github.com/apache/arrow/pull/8403#discussion_r502044409
##########
File path: cpp/src/parquet/arrow/arrow_reader_writer_test.cc
##########
@@ -2689,6 +2689,26 @@ TEST(TestArrowReadWrite, DictionaryColumnChunkedWrite) {
::arrow::AssertTablesEqual(*expected_table, *result, false);
}
+TEST(TestArrowReadWrite, NonUniqueDictionaryValues) {
+ // ARROW-10237
+ auto dict_with_dupes = ArrayFromJSON(::arrow::utf8(), R"(["a", "a", "b"])");
+ auto indices = ArrayFromJSON(::arrow::int32(), R"([0, 1, 0, 1])");
Review comment:
You might want to check all possible index values.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8403:
URL: https://github.com/apache/arrow/pull/8403#issuecomment-705808504
https://issues.apache.org/jira/browse/ARROW-10237
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8403:
URL: https://github.com/apache/arrow/pull/8403#issuecomment-705808504
https://issues.apache.org/jira/browse/ARROW-10237
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz closed pull request #8403: ARROW-10237: [C++] Duplicate dict values cause corrupt parquet
Posted by GitBox <gi...@apache.org>.
bkietz closed pull request #8403:
URL: https://github.com/apache/arrow/pull/8403
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org