You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2025/11/25 14:44:27 UTC
[PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
pitrou opened a new pull request, #48252:
URL: https://github.com/apache/arrow/pull/48252
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how to contribute here:
* [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
Please remove this line and the above text before creating your pull request.
### Rationale for this change
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
**This PR includes breaking changes to public APIs.** (If there are any breaking changes to public APIs, please explain which changes are breaking. If not, you can remove this.)
**This PR contains a "Critical Fix".** (If the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld), please provide explanation. If not, you can remove this.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #48252:
URL: https://github.com/apache/arrow/pull/48252#discussion_r2576093506
##########
cpp/src/arrow/csv/fuzz.cc:
##########
@@ -42,10 +42,11 @@ Status FuzzCsvReader(const uint8_t* data, int64_t size) {
auto read_options = ReadOptions::Defaults();
// Make chunking more likely
- read_options.block_size = 4096;
+ read_options.block_size = 1000;
auto parse_options = ParseOptions::Defaults();
auto convert_options = ConvertOptions::Defaults();
convert_options.auto_dict_encode = true;
+ convert_options.auto_dict_max_cardinality = 50;
Review Comment:
The `block_size` one is to increase the likelihood of chunking and the number of chunks, to exercise chunked reading and parallelization more. The `auto_dict_max_cardinality` just explicitly sets to the default value, so it's really a no-op but it signals a knob that we might want to turn.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #48252:
URL: https://github.com/apache/arrow/pull/48252#discussion_r2560284867
##########
cpp/src/arrow/testing/random.cc:
##########
@@ -65,7 +65,7 @@ template <typename ValueType, typename DistributionType>
struct GeneratorFactory {
GeneratorFactory(ValueType min, ValueType max) : min_(min), max_(max) {}
- auto operator()(pcg32_fast* rng) const {
+ auto operator()(pcg32* rng) const {
Review Comment:
It turns out `pcg32_fast` is not high quality. When used with `RandomArrayGenerator::Strings`, the first string character would very often be `A`...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3586068985
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #48252:
URL: https://github.com/apache/arrow/pull/48252
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "zanmato1984 (via GitHub)" <gi...@apache.org>.
zanmato1984 commented on code in PR #48252:
URL: https://github.com/apache/arrow/pull/48252#discussion_r2576067421
##########
cpp/src/arrow/csv/fuzz.cc:
##########
@@ -42,10 +42,11 @@ Status FuzzCsvReader(const uint8_t* data, int64_t size) {
auto read_options = ReadOptions::Defaults();
// Make chunking more likely
- read_options.block_size = 4096;
+ read_options.block_size = 1000;
auto parse_options = ParseOptions::Defaults();
auto convert_options = ConvertOptions::Defaults();
convert_options.auto_dict_encode = true;
+ convert_options.auto_dict_max_cardinality = 50;
Review Comment:
Why do we need these changes?
##########
cpp/src/arrow/csv/generate_fuzz_corpus.cc:
##########
@@ -0,0 +1,204 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// A command line executable that generates a bunch of valid IPC files
+// containing example record batches. Those are used as fuzzing seeds
+// to make fuzzing more efficient.
+
+#include <cstdlib>
+#include <functional>
+#include <iostream>
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "arrow/array.h"
+#include "arrow/array/util.h"
+#include "arrow/compute/cast.h"
+#include "arrow/csv/options.h"
+#include "arrow/csv/writer.h"
+#include "arrow/io/file.h"
+#include "arrow/io/memory.h"
+#include "arrow/ipc/writer.h"
+#include "arrow/json/from_string.h"
+#include "arrow/record_batch.h"
+#include "arrow/result.h"
+#include "arrow/testing/random.h"
+#include "arrow/util/io_util.h"
+
+namespace arrow::csv {
+
+using ::arrow::internal::CreateDir;
+using ::arrow::internal::PlatformFilename;
+using ::arrow::json::ArrayFromJSONString;
+
+Result<std::shared_ptr<Buffer>> WriteRecordBatch(
+ const std::shared_ptr<RecordBatch>& batch, const WriteOptions& options) {
+ ARROW_ASSIGN_OR_RAISE(auto sink, io::BufferOutputStream::Create(1024));
+ ARROW_ASSIGN_OR_RAISE(auto writer, MakeCSVWriter(sink.get(), batch->schema(), options));
+ RETURN_NOT_OK(writer->WriteRecordBatch(*batch));
+ RETURN_NOT_OK(writer->Close());
+ return sink->Finish();
+}
+
+Result<std::shared_ptr<RecordBatch>> MakeBatch(
+ std::function<Result<std::shared_ptr<Array>>(int64_t length, double null_probability)>
+ array_factory,
+ int64_t length) {
+ ArrayVector columns;
+ FieldVector fields;
+
+ struct ColumnSpec {
+ std::string name;
+ double null_probability;
+ };
+ for (auto spec : {ColumnSpec{"with_nulls", 0.2}, ColumnSpec{"without_nulls", 0.0}}) {
+ ARROW_ASSIGN_OR_RAISE(std::shared_ptr<Array> column,
+ array_factory(length, spec.null_probability));
+ columns.push_back(column);
+ fields.push_back(field(spec.name, column->type()));
+ }
+ return RecordBatch::Make(schema(std::move(fields)), length, std::move(columns));
+}
+
+Result<RecordBatchVector> Batches() {
+ ::arrow::random::RandomArrayGenerator gen(/*seed=*/42);
+ RecordBatchVector batches;
+
+ auto append_batch = [&](auto array_factory, int64_t length) -> Status {
+ ARROW_ASSIGN_OR_RAISE(auto batch, MakeBatch(array_factory, length));
+ batches.push_back(batch);
+ return Status::OK();
+ };
+
+ // Ideally, we should exercise all possible inference kinds (see inference_internal.h)
+ auto make_nulls = [&](int64_t length, double null_probability) {
+ return MakeArrayOfNull(null(), length);
+ };
+ auto make_ints = [&](int64_t length, double null_probability) {
+ return gen.Int64(length, /*min=*/-1'000'000, /*max=*/1'000'000, null_probability);
+ };
+ auto make_floats = [&](int64_t length, double null_probability) {
+ return gen.Float64(length, /*min=*/-100.0, /*max=*/100.0, null_probability);
+ };
+ auto make_booleans = [&](int64_t length, double null_probability) {
+ return gen.Boolean(length, /*true_probability=*/0.8, null_probability);
+ };
+ auto make_dates = [&](int64_t length, double null_probability) {
+ return gen.Date64(length, /*min=*/1, /*max=*/365 * 60, null_probability);
+ };
+ auto make_times = [&](int64_t length, double null_probability) {
+ return gen.Int32(length, /*min=*/0, /*max=*/86399, null_probability)
+ ->View(time32(TimeUnit::SECOND));
+ };
+
+ std::string timezone;
+ auto make_timestamps = [&](int64_t length, double null_probability) {
+ return gen.Int64(length, /*min=*/1, /*max=*/1764079190, null_probability)
+ ->View(timestamp(TimeUnit::SECOND, timezone));
+ };
+ auto make_timestamps_ns = [&](int64_t length, double null_probability) {
+ return gen
+ .Int64(length, /*min=*/1, /*max=*/1764079190LL * 1'000'000'000, null_probability)
+ ->View(timestamp(TimeUnit::NANO, timezone));
+ };
+
+ auto make_strings = [&](int64_t length, double null_probability) {
+ return gen.String(length, /*min_length=*/3, /*max_length=*/15, null_probability);
+ };
+ auto make_string_with_repeats = [&](int64_t length, double null_probability) {
+ // `unique` should be less than `auto_dict_max_cardinality` in fuzz target
+ return gen.StringWithRepeats(length, /*unique=*/10, /*min_length=*/3,
+ /*max_length=*/15, null_probability);
+ };
+
+ RETURN_NOT_OK(append_batch(make_nulls, /*length=*/2000));
+ RETURN_NOT_OK(append_batch(make_ints, /*length=*/500));
+ RETURN_NOT_OK(append_batch(make_floats, /*length=*/150));
+ RETURN_NOT_OK(append_batch(make_booleans, /*length=*/500));
+
+ RETURN_NOT_OK(append_batch(make_dates, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_times, /*length=*/400));
+ timezone = "";
+ RETURN_NOT_OK(append_batch(make_timestamps, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_timestamps_ns, /*length=*/100));
+ // Will generate timestamps with a "Z" suffix
+ timezone = "UTC";
+ RETURN_NOT_OK(append_batch(make_timestamps, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_timestamps_ns, /*length=*/100));
+ // Will generate timestamps with a "+0100" or "+0200" suffix
+ timezone = "Europe/Paris";
+ RETURN_NOT_OK(append_batch(make_timestamps, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_timestamps_ns, /*length=*/100));
+
+ RETURN_NOT_OK(append_batch(make_strings, /*length=*/300));
+ RETURN_NOT_OK(append_batch(make_string_with_repeats, /*length=*/300));
+ // XXX Cannot add non-UTF8 binary as the CSV writer doesn't support writing it
+
+ return batches;
+}
+
+Status DoMain(const std::string& out_dir) {
+ ARROW_ASSIGN_OR_RAISE(auto dir_fn, PlatformFilename::FromString(out_dir));
+ RETURN_NOT_OK(CreateDir(dir_fn));
+
+ int sample_num = 1;
+ auto sample_name = [&]() -> std::string {
+ return "csv-file-" + std::to_string(sample_num++);
+ };
+
+ ARROW_ASSIGN_OR_RAISE(auto batches, Batches());
+
+ auto options = WriteOptions::Defaults();
+ RETURN_NOT_OK(options.Validate());
+
+ for (const auto& batch : batches) {
+ RETURN_NOT_OK(batch->ValidateFull());
+ ARROW_ASSIGN_OR_RAISE(auto buffer, WriteRecordBatch(batch, options));
+
+ ARROW_ASSIGN_OR_RAISE(auto sample_fn, dir_fn.Join(sample_name()));
+ std::cerr << sample_fn.ToString() << std::endl;
Review Comment:
Why use standard error rater than standard out?
##########
cpp/build-support/fuzzing/generate_corpuses.sh:
##########
@@ -71,7 +71,7 @@ rm -rf ${PANDAS_DIR}
git clone --depth=1 https://github.com/pandas-dev/pandas ${PANDAS_DIR}
rm -rf ${CORPUS_DIR}
-mkdir -p ${CORPUS_DIR}
Review Comment:
So before this change, we merely create an empty directory?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #48252:
URL: https://github.com/apache/arrow/pull/48252#discussion_r2576104638
##########
cpp/src/arrow/csv/fuzz.cc:
##########
@@ -42,10 +42,11 @@ Status FuzzCsvReader(const uint8_t* data, int64_t size) {
auto read_options = ReadOptions::Defaults();
// Make chunking more likely
- read_options.block_size = 4096;
+ read_options.block_size = 1000;
auto parse_options = ParseOptions::Defaults();
auto convert_options = ConvertOptions::Defaults();
convert_options.auto_dict_encode = true;
+ convert_options.auto_dict_max_cardinality = 50;
Review Comment:
For the record, most files generated by this PR are 5-10 kB in size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3580718920
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #48252:
URL: https://github.com/apache/arrow/pull/48252#discussion_r2576107429
##########
cpp/src/arrow/csv/generate_fuzz_corpus.cc:
##########
@@ -0,0 +1,204 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// A command line executable that generates a bunch of valid IPC files
+// containing example record batches. Those are used as fuzzing seeds
+// to make fuzzing more efficient.
+
+#include <cstdlib>
+#include <functional>
+#include <iostream>
+#include <memory>
+#include <string>
+#include <vector>
+
+#include "arrow/array.h"
+#include "arrow/array/util.h"
+#include "arrow/compute/cast.h"
+#include "arrow/csv/options.h"
+#include "arrow/csv/writer.h"
+#include "arrow/io/file.h"
+#include "arrow/io/memory.h"
+#include "arrow/ipc/writer.h"
+#include "arrow/json/from_string.h"
+#include "arrow/record_batch.h"
+#include "arrow/result.h"
+#include "arrow/testing/random.h"
+#include "arrow/util/io_util.h"
+
+namespace arrow::csv {
+
+using ::arrow::internal::CreateDir;
+using ::arrow::internal::PlatformFilename;
+using ::arrow::json::ArrayFromJSONString;
+
+Result<std::shared_ptr<Buffer>> WriteRecordBatch(
+ const std::shared_ptr<RecordBatch>& batch, const WriteOptions& options) {
+ ARROW_ASSIGN_OR_RAISE(auto sink, io::BufferOutputStream::Create(1024));
+ ARROW_ASSIGN_OR_RAISE(auto writer, MakeCSVWriter(sink.get(), batch->schema(), options));
+ RETURN_NOT_OK(writer->WriteRecordBatch(*batch));
+ RETURN_NOT_OK(writer->Close());
+ return sink->Finish();
+}
+
+Result<std::shared_ptr<RecordBatch>> MakeBatch(
+ std::function<Result<std::shared_ptr<Array>>(int64_t length, double null_probability)>
+ array_factory,
+ int64_t length) {
+ ArrayVector columns;
+ FieldVector fields;
+
+ struct ColumnSpec {
+ std::string name;
+ double null_probability;
+ };
+ for (auto spec : {ColumnSpec{"with_nulls", 0.2}, ColumnSpec{"without_nulls", 0.0}}) {
+ ARROW_ASSIGN_OR_RAISE(std::shared_ptr<Array> column,
+ array_factory(length, spec.null_probability));
+ columns.push_back(column);
+ fields.push_back(field(spec.name, column->type()));
+ }
+ return RecordBatch::Make(schema(std::move(fields)), length, std::move(columns));
+}
+
+Result<RecordBatchVector> Batches() {
+ ::arrow::random::RandomArrayGenerator gen(/*seed=*/42);
+ RecordBatchVector batches;
+
+ auto append_batch = [&](auto array_factory, int64_t length) -> Status {
+ ARROW_ASSIGN_OR_RAISE(auto batch, MakeBatch(array_factory, length));
+ batches.push_back(batch);
+ return Status::OK();
+ };
+
+ // Ideally, we should exercise all possible inference kinds (see inference_internal.h)
+ auto make_nulls = [&](int64_t length, double null_probability) {
+ return MakeArrayOfNull(null(), length);
+ };
+ auto make_ints = [&](int64_t length, double null_probability) {
+ return gen.Int64(length, /*min=*/-1'000'000, /*max=*/1'000'000, null_probability);
+ };
+ auto make_floats = [&](int64_t length, double null_probability) {
+ return gen.Float64(length, /*min=*/-100.0, /*max=*/100.0, null_probability);
+ };
+ auto make_booleans = [&](int64_t length, double null_probability) {
+ return gen.Boolean(length, /*true_probability=*/0.8, null_probability);
+ };
+ auto make_dates = [&](int64_t length, double null_probability) {
+ return gen.Date64(length, /*min=*/1, /*max=*/365 * 60, null_probability);
+ };
+ auto make_times = [&](int64_t length, double null_probability) {
+ return gen.Int32(length, /*min=*/0, /*max=*/86399, null_probability)
+ ->View(time32(TimeUnit::SECOND));
+ };
+
+ std::string timezone;
+ auto make_timestamps = [&](int64_t length, double null_probability) {
+ return gen.Int64(length, /*min=*/1, /*max=*/1764079190, null_probability)
+ ->View(timestamp(TimeUnit::SECOND, timezone));
+ };
+ auto make_timestamps_ns = [&](int64_t length, double null_probability) {
+ return gen
+ .Int64(length, /*min=*/1, /*max=*/1764079190LL * 1'000'000'000, null_probability)
+ ->View(timestamp(TimeUnit::NANO, timezone));
+ };
+
+ auto make_strings = [&](int64_t length, double null_probability) {
+ return gen.String(length, /*min_length=*/3, /*max_length=*/15, null_probability);
+ };
+ auto make_string_with_repeats = [&](int64_t length, double null_probability) {
+ // `unique` should be less than `auto_dict_max_cardinality` in fuzz target
+ return gen.StringWithRepeats(length, /*unique=*/10, /*min_length=*/3,
+ /*max_length=*/15, null_probability);
+ };
+
+ RETURN_NOT_OK(append_batch(make_nulls, /*length=*/2000));
+ RETURN_NOT_OK(append_batch(make_ints, /*length=*/500));
+ RETURN_NOT_OK(append_batch(make_floats, /*length=*/150));
+ RETURN_NOT_OK(append_batch(make_booleans, /*length=*/500));
+
+ RETURN_NOT_OK(append_batch(make_dates, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_times, /*length=*/400));
+ timezone = "";
+ RETURN_NOT_OK(append_batch(make_timestamps, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_timestamps_ns, /*length=*/100));
+ // Will generate timestamps with a "Z" suffix
+ timezone = "UTC";
+ RETURN_NOT_OK(append_batch(make_timestamps, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_timestamps_ns, /*length=*/100));
+ // Will generate timestamps with a "+0100" or "+0200" suffix
+ timezone = "Europe/Paris";
+ RETURN_NOT_OK(append_batch(make_timestamps, /*length=*/200));
+ RETURN_NOT_OK(append_batch(make_timestamps_ns, /*length=*/100));
+
+ RETURN_NOT_OK(append_batch(make_strings, /*length=*/300));
+ RETURN_NOT_OK(append_batch(make_string_with_repeats, /*length=*/300));
+ // XXX Cannot add non-UTF8 binary as the CSV writer doesn't support writing it
+
+ return batches;
+}
+
+Status DoMain(const std::string& out_dir) {
+ ARROW_ASSIGN_OR_RAISE(auto dir_fn, PlatformFilename::FromString(out_dir));
+ RETURN_NOT_OK(CreateDir(dir_fn));
+
+ int sample_num = 1;
+ auto sample_name = [&]() -> std::string {
+ return "csv-file-" + std::to_string(sample_num++);
+ };
+
+ ARROW_ASSIGN_OR_RAISE(auto batches, Batches());
+
+ auto options = WriteOptions::Defaults();
+ RETURN_NOT_OK(options.Validate());
+
+ for (const auto& batch : batches) {
+ RETURN_NOT_OK(batch->ValidateFull());
+ ARROW_ASSIGN_OR_RAISE(auto buffer, WriteRecordBatch(batch, options));
+
+ ARROW_ASSIGN_OR_RAISE(auto sample_fn, dir_fn.Join(sample_name()));
+ std::cerr << sample_fn.ToString() << std::endl;
Review Comment:
No precise reason, this is the [same thing](https://github.com/apache/arrow/blob/8ca883f2ffcffd64b662cc8c6e42a78159a0d877/cpp/src/arrow/ipc/generate_fuzz_corpus.cc#L145-L147) we're doing in other fuzz corpus generators.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3580735777
Revision: bcce6c56c2edbaf5a6417e99ac43729857889c4f
Submitted crossbow builds: [ursacomputing/crossbow @ actions-e5f01b72d0](https://github.com/ursacomputing/crossbow/branches/all?query=actions-e5f01b72d0)
|Task|Status|
|----|------|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions/runs/19701043071/job/56437132237)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042861/job/56437131503)|
|example-cpp-tutorial|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042491/job/56437129930)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042195/job/56437128990)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19701041977/job/56437128000)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042881/job/56437131615)|
|test-cuda-cpp-ubuntu-22.04-cuda-11.7.1|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042732/job/56437131385)|
|test-debian-12-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042276/job/56437129211)|
|test-debian-12-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042563/job/56437133660)|
|test-fedora-42-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19701041640/job/56437127047)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19701041956/job/56437127859)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042328/job/56437129412)|
|test-ubuntu-22.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/19701041820/job/56437127433)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/19701041910/job/56437127727)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042712/job/56437131084)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042571/job/56437130254)|
|test-ubuntu-24.04-cpp-bundled-offline|[](https://github.com/ursacomputing/crossbow/actions/runs/19701043245/job/56437132682)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/19701043407/job/56437133236)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/19701043218/job/56437132617)|
|test-ubuntu-24.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/19701042018/job/56437128430)|
|test-ubuntu-24.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/19701043403/job/56437134142)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3586082126
Revision: 7d455965ddce652f77f84a869040c10df87172b2
Submitted crossbow builds: [ursacomputing/crossbow @ actions-09618dfadc](https://github.com/ursacomputing/crossbow/branches/all?query=actions-09618dfadc)
|Task|Status|
|----|------|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042140/job/56558026296)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042454/job/56558025717)|
|example-cpp-tutorial|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042748/job/56558026947)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/19739041830/job/56558023849)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042643/job/56558026617)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/19739041835/job/56558023755)|
|test-cuda-cpp-ubuntu-22.04-cuda-11.7.1|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042161/job/56558024897)|
|test-debian-12-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042036/job/56558024817)|
|test-debian-12-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042280/job/56558025316)|
|test-fedora-42-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042534/job/56558026297)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042918/job/56558027451)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/19739041793/job/56558024278)|
|test-ubuntu-22.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042392/job/56558025669)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042902/job/56558027538)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042935/job/56558027716)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/19739043133/job/56558028054)|
|test-ubuntu-24.04-cpp-bundled-offline|[](https://github.com/ursacomputing/crossbow/actions/runs/19739041970/job/56558024325)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/19739043051/job/56558027741)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/19739042726/job/56558026959)|
|test-ubuntu-24.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/19739043189/job/56558028597)|
|test-ubuntu-24.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/19739041785/job/56558023943)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3595290083
@github-actions crossbow submit *fuzz*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3662731035
After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit a32730c4884171fb9659bc1c0b4d86347a77c529.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/58256602010) has more details. It also includes information about 105 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #48252:
URL: https://github.com/apache/arrow/pull/48252#discussion_r2576085435
##########
cpp/build-support/fuzzing/generate_corpuses.sh:
##########
@@ -71,7 +71,7 @@ rm -rf ${PANDAS_DIR}
git clone --depth=1 https://github.com/pandas-dev/pandas ${PANDAS_DIR}
rm -rf ${CORPUS_DIR}
-mkdir -p ${CORPUS_DIR}
Review Comment:
We were also copying a bunch of pre-existing CSV files into it, see `cp` commands below :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48251: [C++][CI] Add CSV fuzzing seed corpus generator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #48252:
URL: https://github.com/apache/arrow/pull/48252#issuecomment-3595296950
Revision: 2f092c29a39f44bb57f5ced25b876e06ad13639e
Submitted crossbow builds: [ursacomputing/crossbow @ actions-805c4b6939](https://github.com/ursacomputing/crossbow/branches/all?query=actions-805c4b6939)
|Task|Status|
|----|------|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/19816467815/job/56768696705)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org