You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/12/22 18:45:27 UTC
[GitHub] [spark] steveloughran opened a new pull request, #39185: SPARK-41551. Dynamic/absolute path support in PathOutputCommitters
steveloughran opened a new pull request, #39185:
URL: https://github.com/apache/spark/pull/39185
### What changes were proposed in this pull request?
Follow on to SPARK-40034.
Dynamic partitioning though the PathOutputCommitProtocol needs to add the dirs to the superclass's partition list else the partition delete doesn't
take place.
Fix:
* add an addPartition() method subclasses can use
* add a getPartitions method to return an immutable
copy of the list for testing.
* add tests to verify all of this.
Also fix newTaskTempFileAbsPath to return a path, irrespective of committer type.
In dynamic mode, because the parent dir of an absolute path is deleted, there's a safety check to reject any requests for a file in a parent dir. This
is something which could be pulled up to HadoopMapReduceCommitProtocol because it needs the same check, if the risk is considered realistic.
The patch now downgrades from failing on dynamic partitioning if the committer doesn't declare it supports it to printing a warning. Why this? well, it
*does* work through the s3a committers, it's just O(data). If someone does want to do INSERT OVERWRITE then they can be allowed to, just warned about
it. The outcome will be correct except in the case of: "if the driver fails partway through dir rename, only some of the files will be there"
Google GCS has that same non-atomic rename issue. But: even on an FS with atomic dir rename, any job which fails partway through the overwrite process
is going to leave the fs in an inconsistent state, such as
* some parts with new data, some parts not yet overwritten
* a directory deleted and the new data not instantiated
So it's not that much worse.
The patch tries to update the protocol spec in HadoopMapReduceCommitProtocol to cover both newFileAbsPath() semantics/commit and failure modes of
dynamic partition commit.
### Why are the changes needed?
* dynamic partition jobs aren't actually incorrect through the s3 committers, just slow, so if someone really wants it they should be free to.
* the code in SPARK-40034 was incomplete. The new tests show that it works now.
* the `newFileAbsPath()` code is required of all committers, even though (thankfully) it is not used much.
This is an attempt to do a minimal commit for complete functionality, with no attempt to improve performance with parallel dir setup, rename, etc.
That'd really be damage limitation: if you want performance in cloud, use a manifest format.
### Does this PR introduce _any_ user-facing change?
Updates the cloud docs to say that dynamic partition overwrite does work
everywhere, just may be really slow.
### How was this patch tested?
New unit tests; run through spark and hadoop 3.3.5 RC0
One test area which is not covered is "does the absolute path mapping work store the correct paths and marshal them correctly?"
I've just realised that I can test this without adding an accessor for the map just by calling `commitTask()` and parsing the `TaskCommitMessage` which comes back.
Shall I add that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] steveloughran commented on pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
Posted by GitBox <gi...@apache.org>.
steveloughran commented on PR #39185:
URL: https://github.com/apache/spark/pull/39185#issuecomment-1364000436
1. I think i should add a test which verifies the abs path temp file is always under .spark-staging *and* includes the task attempt id in it.
1. jobs created through RDDs don't get unique IDs across processes, they just get the RDD counter. so > 1 job writing to same dest may really ruin the lives of others even when updating different partitions. I can fix that here or file a separate JIRA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] steveloughran commented on pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #39185:
URL: https://github.com/apache/spark/pull/39185#issuecomment-1448463706
I've created a more minimal PR with the earlier production code changes and the newer tests: #40221 ; if all is happy there it can go in while this new protocol variant can be considered/stabilised.
Once spark switches to 3.3.5 as the hadoop 3 version, testing all this stuff becomes easier, as the ManifestCommitter will always be there, which tests can then code for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] steveloughran commented on pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #39185:
URL: https://github.com/apache/spark/pull/39185#issuecomment-1416160287
latest commit adds parallel file/dir preparation and rename, for performance on slow/very slow cloud stores.
As a consequence, it now needs better tests, FileOutputCommitter can be used for the work, though it doesn't do the other feature: IOStatistics collection
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] steveloughran commented on pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #39185:
URL: https://github.com/apache/spark/pull/39185#issuecomment-1419461503
latest pr will save to a report dir aggregate IOStats collected from the task attempts, in addition any _SUCCESS reports created by the inner committer. This is to align later with collection of context IOStats, where all stream read/write stats are collected.
For now, a json summary of the stats collected/reported by the commmitters are collected, for example
```
{
"counters" : {
"committer_commit_job" : 1,
"op_msync" : 1,
"op_msync.failures" : 0,
"job_stage_optional_validate_output" : 0,
"job_stage_create_target_dirs" : 1,
"op_load_manifest.failures" : 0,
"op_rename.failures" : 0,
"op_load_all_manifests" : 1,
"job_stage_load_manifests" : 1,
"committer_commit_job.failures" : 0,
"task_stage_save_task_manifest" : 3,
"job_stage_rename_files" : 1,
"committer_bytes_committed" : 9,
"op_create_one_directory.failures" : 0,
"op_mkdirs.failures" : 0,
"committer_files_committed" : 1,
"store_io_rate_limited.failures" : 0,
"op_load_all_manifests.failures" : 0,
"committer_task_manifest_file_size" : 0,
"task_stage_save_manifest" : 2,
"task_stage_scan_directory" : 3,
"task_stage_setup.failures" : 0,
"op_is_file.failures" : 0,
"op_create_directories.failures" : 0,
"op_create_one_directory" : 0,
"committer_task_file_count" : 3,
"task_stage_save_manifest.failures" : 0,
"committer_task_directory_depth" : 6,
"job_stage_setup.failures" : 0,
"job_stage_load_manifests.failures" : 0,
"op_directory_scan" : 0,
"op_rename" : 3,
"job_stage_abort" : 0,
"op_get_file_status" : 9,
"commit_file_rename_recovered" : 0,
"store_io_rate_limited" : 0,
"op_delete_file_under_destination.failures" : 0,
"op_prepare_dir_ancestors" : 0,
"job_stage_abort.failures" : 0,
"op_is_directory.failures" : 0,
"job_stage_optional_validate_output.failures" : 0,
"task_stage_setup" : 3,
"commit_file_rename.failures" : 0,
"op_delete" : 6,
"op_delete_file_under_destination" : 0,
"job_stage_save_success_marker" : 1,
"commit_file_rename" : 1,
"op_mkdir_returned_false" : 0,
"op_directory_scan.failures" : 0,
"object_list_request.failures" : 0,
"object_continue_list_request.failures" : 0,
"op_load_manifest" : 1,
"committer_tasks_completed" : 2,
"job_stage_rename_files.failures" : 0,
"op_get_file_status.failures" : 8,
"task_stage_scan_directory.failures" : 0,
"op_delete.failures" : 0,
"object_list_request" : 0,
"op_list_status" : 8,
"task_stage_abort_task.failures" : 0,
"job_stage_setup" : 1,
"task_stage_commit" : 3,
"committer_tasks_failed" : 0,
"task_stage_commit.failures" : 0,
"task_stage_abort_task" : 0,
"committer_task_file_size" : 27,
"job_stage_create_target_dirs.failures" : 0,
"committer_task_directory_count" : 3,
"object_continue_list_request" : 0,
"job_stage_save_success_marker.failures" : 0,
"op_create_directories" : 1,
"op_mkdir_returned_false.failures" : 0,
"task_stage_save_task_manifest.failures" : 0,
"job_stage_cleanup.failures" : 0,
"job_stage_cleanup" : 1,
"op_mkdirs" : 6,
"op_is_directory" : 0,
"op_prepare_dir_ancestors.failures" : 0,
"op_is_file" : 0,
"op_list_status.failures" : 0
},
"gauges" : { },
"minimums" : {
"task_stage_commit.min" : 1,
"task_stage_save_task_manifest.min" : 21,
"op_create_directories.failures.min" : -1,
"op_is_directory.failures.min" : -1,
"job_stage_abort.min" : -1,
"object_list_request.min" : -1,
"task_stage_save_task_manifest.failures.min" : -1,
"object_continue_list_request.failures.min" : -1,
"op_prepare_dir_ancestors.failures.min" : -1,
"object_list_request.failures.min" : -1,
"op_directory_scan.min" : -1,
"job_stage_cleanup.min" : 14,
"op_list_status.min" : 0,
"op_mkdirs.failures.min" : -1,
"task_stage_setup.failures.min" : -1,
"job_stage_abort.failures.min" : -1,
"committer_bytes_committed" : -1,
"job_stage_rename_files.failures.min" : -1,
"op_is_file.failures.min" : -1,
"committer_files_committed" : -1,
"job_stage_rename_files.min" : 13,
"op_load_all_manifests.min" : 12,
"op_rename.failures.min" : -1,
"op_delete_file_under_destination.failures.min" : -1,
"task_stage_setup.min" : 29,
"job_stage_create_target_dirs.min" : 14,
"op_msync.failures.min" : -1,
"commit_file_rename.failures.min" : -1,
"job_stage_cleanup.failures.min" : -1,
"job_stage_save_success_marker.failures.min" : -1,
"store_io_rate_limited.min" : -1,
"job_stage_save_success_marker.min" : 28,
"committer_task_manifest_file_size" : -1,
"job_stage_create_target_dirs.failures.min" : -1,
"op_load_manifest.failures.min" : -1,
"committer_commit_job.min" : 83,
"committer_commit_job.failures.min" : -1,
"op_delete.min" : 0,
"op_prepare_dir_ancestors.min" : -1,
"committer_task_file_count" : 1,
"job_stage_setup.min" : 74,
"committer_task_directory_depth" : 2,
"op_get_file_status.min" : 0,
"job_stage_load_manifests.min" : 13,
"op_load_all_manifests.failures.min" : -1,
"job_stage_optional_validate_output.failures.min" : -1,
"op_get_file_status.failures.min" : 0,
"op_create_one_directory.failures.min" : -1,
"op_load_manifest.min" : 2,
"op_create_one_directory.min" : -1,
"commit_file_rename_recovered" : -1,
"op_mkdirs.min" : 13,
"task_stage_scan_directory.failures.min" : -1,
"op_msync.min" : 0,
"op_mkdir_returned_false.min" : -1,
"op_is_file.min" : -1,
"store_io_rate_limited.failures.min" : -1,
"task_stage_save_manifest.failures.min" : -1,
"op_list_status.failures.min" : -1,
"task_stage_abort_task.min" : -1,
"op_directory_scan.failures.min" : -1,
"committer_tasks_completed" : -1,
"commit_file_rename.min" : 0,
"op_delete_file_under_destination.min" : -1,
"object_continue_list_request.min" : -1,
"job_stage_load_manifests.failures.min" : -1,
"committer_tasks_failed" : -1,
"committer_task_file_size" : 9,
"job_stage_optional_validate_output.min" : -1,
"job_stage_setup.failures.min" : -1,
"committer_task_directory_count" : 1,
"op_delete.failures.min" : -1,
"task_stage_commit.failures.min" : -1,
"op_create_directories.min" : 13,
"task_stage_abort_task.failures.min" : -1,
"op_mkdir_returned_false.failures.min" : -1,
"op_is_directory.min" : -1,
"task_stage_scan_directory.min" : 1,
"task_stage_save_manifest.min" : 21,
"op_rename.min" : 0
},
"maximums" : {
"job_stage_cleanup.max" : 14,
"job_stage_load_manifests.max" : 13,
"op_prepare_dir_ancestors.max" : -1,
"op_load_all_manifests.failures.max" : -1,
"op_mkdir_returned_false.failures.max" : -1,
"job_stage_rename_files.failures.max" : -1,
"op_create_one_directory.max" : -1,
"op_is_file.failures.max" : -1,
"op_msync.failures.max" : -1,
"op_is_directory.failures.max" : -1,
"op_msync.max" : 0,
"job_stage_save_success_marker.max" : 28,
"object_list_request.failures.max" : -1,
"committer_bytes_committed" : -1,
"op_prepare_dir_ancestors.failures.max" : -1,
"committer_files_committed" : -1,
"task_stage_save_task_manifest.max" : 28,
"job_stage_optional_validate_output.max" : -1,
"job_stage_setup.max" : 74,
"op_directory_scan.max" : -1,
"job_stage_optional_validate_output.failures.max" : -1,
"op_mkdirs.max" : 51,
"task_stage_abort_task.max" : -1,
"committer_task_manifest_file_size" : -1,
"op_get_file_status.failures.max" : 0,
"op_create_one_directory.failures.max" : -1,
"commit_file_rename.max" : 0,
"op_delete_file_under_destination.max" : -1,
"op_create_directories.failures.max" : -1,
"committer_task_file_count" : 1,
"op_rename.failures.max" : -1,
"committer_task_directory_depth" : 2,
"op_delete_file_under_destination.failures.max" : -1,
"job_stage_abort.max" : -1,
"commit_file_rename.failures.max" : -1,
"job_stage_create_target_dirs.failures.max" : -1,
"op_is_file.max" : -1,
"commit_file_rename_recovered" : -1,
"job_stage_save_success_marker.failures.max" : -1,
"op_directory_scan.failures.max" : -1,
"op_delete.max" : 2,
"object_list_request.max" : -1,
"task_stage_setup.max" : 29,
"op_get_file_status.max" : 0,
"committer_commit_job.max" : 83,
"op_delete.failures.max" : -1,
"object_continue_list_request.max" : -1,
"op_is_directory.max" : -1,
"task_stage_save_manifest.max" : 21,
"task_stage_scan_directory.failures.max" : -1,
"store_io_rate_limited.max" : -1,
"op_list_status.failures.max" : -1,
"job_stage_create_target_dirs.max" : 14,
"job_stage_cleanup.failures.max" : -1,
"op_mkdir_returned_false.max" : -1,
"op_load_manifest.max" : 2,
"committer_tasks_completed" : -1,
"op_load_manifest.failures.max" : -1,
"store_io_rate_limited.failures.max" : -1,
"committer_commit_job.failures.max" : -1,
"task_stage_save_manifest.failures.max" : -1,
"op_create_directories.max" : 13,
"task_stage_commit.failures.max" : -1,
"op_mkdirs.failures.max" : -1,
"task_stage_setup.failures.max" : -1,
"task_stage_abort_task.failures.max" : -1,
"committer_tasks_failed" : -1,
"op_list_status.max" : 1,
"committer_task_file_size" : 9,
"task_stage_scan_directory.max" : 1,
"job_stage_rename_files.max" : 13,
"op_rename.max" : 0,
"committer_task_directory_count" : 1,
"op_load_all_manifests.max" : 12,
"job_stage_load_manifests.failures.max" : -1,
"job_stage_abort.failures.max" : -1,
"job_stage_setup.failures.max" : -1,
"object_continue_list_request.failures.max" : -1,
"task_stage_save_task_manifest.failures.max" : -1,
"task_stage_commit.max" : 22
},
"meanstatistics" : {
"commit_file_rename.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"task_stage_save_manifest.mean" : {
"samples" : 2,
"sum" : 42
},
"job_stage_optional_validate_output.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_save_success_marker.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_rename.mean" : {
"samples" : 3,
"sum" : 0
},
"op_load_manifest.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_is_directory.mean" : {
"samples" : 0,
"sum" : 0
},
"op_create_one_directory.mean" : {
"samples" : 0,
"sum" : 0
},
"commit_file_rename.mean" : {
"samples" : 1,
"sum" : 0
},
"op_rename.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_mkdir_returned_false.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_rename_files.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_load_manifests.mean" : {
"samples" : 1,
"sum" : 13
},
"committer_bytes_committed" : {
"samples" : 0,
"sum" : 0
},
"op_delete.mean" : {
"samples" : 6,
"sum" : 2
},
"job_stage_cleanup.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_files_committed" : {
"samples" : 0,
"sum" : 0
},
"task_stage_setup.mean" : {
"samples" : 3,
"sum" : 87
},
"task_stage_save_task_manifest.mean" : {
"samples" : 3,
"sum" : 70
},
"op_load_manifest.mean" : {
"samples" : 1,
"sum" : 2
},
"task_stage_abort_task.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"store_io_rate_limited.mean" : {
"samples" : 0,
"sum" : 0
},
"op_mkdir_returned_false.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_task_manifest_file_size" : {
"samples" : 1,
"sum" : 18835
},
"op_is_directory.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_mkdirs.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"task_stage_scan_directory.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"task_stage_setup.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_abort.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_task_file_count" : {
"samples" : 3,
"sum" : 3
},
"object_list_request.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_task_directory_depth" : {
"samples" : 3,
"sum" : 6
},
"task_stage_save_task_manifest.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_is_file.mean" : {
"samples" : 0,
"sum" : 0
},
"op_prepare_dir_ancestors.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"commit_file_rename_recovered" : {
"samples" : 0,
"sum" : 0
},
"task_stage_save_manifest.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_setup.mean" : {
"samples" : 1,
"sum" : 74
},
"object_list_request.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_get_file_status.failures.mean" : {
"samples" : 8,
"sum" : 0
},
"task_stage_commit.mean" : {
"samples" : 5,
"sum" : 47
},
"object_continue_list_request.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_cleanup.mean" : {
"samples" : 1,
"sum" : 14
},
"job_stage_create_target_dirs.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_delete_file_under_destination.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_directory_scan.mean" : {
"samples" : 0,
"sum" : 0
},
"op_get_file_status.mean" : {
"samples" : 1,
"sum" : 0
},
"job_stage_create_target_dirs.mean" : {
"samples" : 1,
"sum" : 14
},
"committer_tasks_completed" : {
"samples" : 0,
"sum" : 0
},
"job_stage_abort.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_load_all_manifests.mean" : {
"samples" : 1,
"sum" : 12
},
"op_create_directories.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_list_status.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_save_success_marker.mean" : {
"samples" : 1,
"sum" : 28
},
"committer_commit_job.mean" : {
"samples" : 1,
"sum" : 83
},
"op_is_file.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_create_one_directory.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_load_all_manifests.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_directory_scan.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_mkdirs.mean" : {
"samples" : 6,
"sum" : 174
},
"store_io_rate_limited.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_load_manifests.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_list_status.mean" : {
"samples" : 8,
"sum" : 4
},
"op_prepare_dir_ancestors.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_commit_job.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_tasks_failed" : {
"samples" : 0,
"sum" : 0
},
"committer_task_file_size" : {
"samples" : 3,
"sum" : 27
},
"job_stage_optional_validate_output.mean" : {
"samples" : 0,
"sum" : 0
},
"committer_task_directory_count" : {
"samples" : 3,
"sum" : 3
},
"op_create_directories.mean" : {
"samples" : 1,
"sum" : 13
},
"task_stage_scan_directory.mean" : {
"samples" : 3,
"sum" : 3
},
"op_delete_file_under_destination.mean" : {
"samples" : 0,
"sum" : 0
},
"object_continue_list_request.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"task_stage_commit.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"task_stage_abort_task.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_rename_files.mean" : {
"samples" : 1,
"sum" : 13
},
"op_delete.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"op_msync.mean" : {
"samples" : 1,
"sum" : 0
},
"op_msync.failures.mean" : {
"samples" : 0,
"sum" : 0
},
"job_stage_setup.failures.mean" : {
"samples" : 0,
"sum" : 0
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
URL: https://github.com/apache/spark/pull/39185
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #39185: [SPARK-41551][SQL] Dynamic/absolute path support in PathOutputCommitters
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39185:
URL: https://github.com/apache/spark/pull/39185#issuecomment-1583660791
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org