You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/21 13:00:00 UTC
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #12838: ARROW-15757: [Python] Missing bindings for existing_data_behavior makes it impossible to maintain old behavior
jorisvandenbossche commented on code in PR #12838:
URL: https://github.com/apache/arrow/pull/12838#discussion_r855156736
##########
python/pyarrow/parquet/__init__.py:
##########
@@ -3116,6 +3136,14 @@ def file_visitor(written_file):
"'basename_template' parameter instead. For "
"usage see `pyarrow.dataset.write_dataset`"),
FutureWarning, stacklevel=2)
+ if existing_data_behavior is not None:
Review Comment:
```suggestion
if existing_data_behavior != "overwrite_or_ignore":
```
? (since that is the default value. Or otherwise we need to change the default value to None and turn that into "overwrite_or_ignore" in the use_legacy_dataset=False code path)
##########
python/pyarrow/parquet/__init__.py:
##########
@@ -3116,6 +3136,14 @@ def file_visitor(written_file):
"'basename_template' parameter instead. For "
"usage see `pyarrow.dataset.write_dataset`"),
FutureWarning, stacklevel=2)
+ if existing_data_behavior is not None:
+ warnings.warn(
+ _DEPR_MSG.format("existing_data_behavior", " Specify "
+ "'use_legacy_dataset=False' while constructing "
+ "the ParquetDataset, and then use the "
+ "'basename_template' parameter instead. For "
Review Comment:
this last part is not relevant here I think (copy paste from partition_filename_cb?)
##########
python/pyarrow/parquet/__init__.py:
##########
@@ -3116,6 +3136,14 @@ def file_visitor(written_file):
"'basename_template' parameter instead. For "
"usage see `pyarrow.dataset.write_dataset`"),
FutureWarning, stacklevel=2)
+ if existing_data_behavior is not None:
+ warnings.warn(
+ _DEPR_MSG.format("existing_data_behavior", " Specify "
+ "'use_legacy_dataset=False' while constructing "
+ "the ParquetDataset, and then use the "
+ "'basename_template' parameter instead. For "
Review Comment:
This could also be an error like the schema/partitioning/use_threads handling above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org