You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "mapleFU (via GitHub)" <gi...@apache.org> on 2023/02/08 08:03:28 UTC
[GitHub] [arrow] mapleFU opened a new issue, #34073: [C++][Parquet] Parquet Fuzzing Enhancement
mapleFU opened a new issue, #34073:
URL: https://github.com/apache/arrow/issues/34073
### Describe the enhancement requested
Currently, Parquet uses LLVM libFuzz and corpus to run fuzzing. Corpus will generate a multi-column file, and copy all valid test-data to the fuzz directory.
However, libFuzz mutate value by byte, though libFuzz uses coverage-based fuzzing, it's hard to generate valid input, like different read and write options, different kinds of encoding. Instead, it's likely to generate some bad-data (lol...)
### Component(s)
C++, Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Parquet Fuzzing Enhancement [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #34073:
URL: https://github.com/apache/arrow/issues/34073#issuecomment-3921683874
This issue has been marked as stale because it has had no activity in the past 2 years. Please remove the stale label or comment below, or this issue will be closed in 14 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][Parquet] Parquet Fuzzing Enhancement [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic closed issue #34073: [C++][Parquet] Parquet Fuzzing Enhancement
URL: https://github.com/apache/arrow/issues/34073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace commented on issue #34073: [C++][Parquet] Parquet Fuzzing Enhancement
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on issue #34073:
URL: https://github.com/apache/arrow/issues/34073#issuecomment-1423064532
You may also want to look at the integration tests we have for the Arrow IPC format for inspiration. This is in `dev/archery/archery/integration`. There are tools there to generate random valid IPC files which we then verify the various implementations can all read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #34073: [C++][Parquet] Parquet Fuzzing Enhancement
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #34073:
URL: https://github.com/apache/arrow/issues/34073#issuecomment-1423539935
Great! `datagen.py` and cross generation helps a lot! Previously I found that Rust's arrow has a cross-verification test, which write from pyarrow and read from it self's implemention. Let me do it step by step: https://github.com/jorgecarleitao/parquet2/pull/104/files#diff-b1ec89bd2022a3ed591256cf38963132122a430d36841741185a65ac4177b249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org