You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by GitBox <gi...@apache.org> on 2023/01/04 15:23:53 UTC
[GitHub] [arrow] pitrou opened a new issue, #15193: [C++][Parquet] Fuzz using several batch sizes
pitrou opened a new issue, #15193:
URL: https://github.com/apache/arrow/issues/15193
### Describe the enhancement requested
Exercising several batch sizes in `parquet::arrow::internal::FuzzReader` may help find more reader/decoder bugs.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed issue #15193: [C++][Parquet] Fuzz using several batch sizes
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #15193: [C++][Parquet] Fuzz using several batch sizes
URL: https://github.com/apache/arrow/issues/15193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #15193: [C++][Parquet] Fuzz using several batch sizes
Posted by GitBox <gi...@apache.org>.
pitrou commented on issue #15193:
URL: https://github.com/apache/arrow/issues/15193#issuecomment-1371143947
Something like this could work:
```diff
diff --git a/cpp/src/parquet/arrow/reader.cc b/cpp/src/parquet/arrow/reader.cc
index b57b5062a..717f49629 100644
--- a/cpp/src/parquet/arrow/reader.cc
+++ b/cpp/src/parquet/arrow/reader.cc
@@ -20,6 +20,7 @@
#include <algorithm>
#include <cstring>
#include <memory>
+#include <optional>
#include <unordered_set>
#include <utility>
#include <vector>
@@ -1385,13 +1386,21 @@ Status FuzzReader(std::unique_ptr<FileReader> reader) {
Status FuzzReader(const uint8_t* data, int64_t size) {
auto buffer = std::make_shared<::arrow::Buffer>(data, size);
- auto file = std::make_shared<::arrow::io::BufferReader>(buffer);
- FileReaderBuilder builder;
- RETURN_NOT_OK(builder.Open(std::move(file)));
+ auto st = Status::OK();
+
+ for (auto batch_size : std::vector<std::optional<int>>{std::nullopt, 1, 13, 300}) {
+ auto file = std::make_shared<::arrow::io::BufferReader>(buffer);
+ FileReaderBuilder builder;
+ RETURN_NOT_OK(builder.Open(std::move(file)));
- std::unique_ptr<FileReader> reader;
- RETURN_NOT_OK(builder.Build(&reader));
- return FuzzReader(std::move(reader));
+ std::unique_ptr<FileReader> reader;
+ RETURN_NOT_OK(builder.Build(&reader));
+ if (batch_size) {
+ reader->set_batch_size(*batch_size);
+ }
+ st &= FuzzReader(std::move(reader));
+ }
+ return st;
}
} // namespace internal
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #15193: [C++][Parquet] Fuzz using several batch sizes
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #15193:
URL: https://github.com/apache/arrow/issues/15193#issuecomment-1409778653
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #15193: [C++][Parquet] Fuzz using several batch sizes
Posted by GitBox <gi...@apache.org>.
mapleFU commented on issue #15193:
URL: https://github.com/apache/arrow/issues/15193#issuecomment-1385853122
I'd like to pickup this issue, but I need to learn fuzz first :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org