You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Xinyu Tan <ta...@apache.org> on 2026/10/23 08:55:52 UTC

[Review Request] HADOOP-17611: Preserve DistCp timestamps after chunk concatenation

Hi Hadoop developers,

I'd like to request a review of a fix for HADOOP-17611, where DistCp's parallel file copy (-blocksperchunk) breaks
modification time preservation.

When -pt is specified, the mapper preserves timestamps on the chunks, but the subsequent concat operation changes the
final file's modification time. This patch restores the source modification and access times after concatenation and
checksum validation, only when TIMES preservation is requested.

The PR adds 14 HDFS contract test cases covering timestamp preservation enabled/disabled, ordinary/direct/atomic
copies, directory scope, and update/skip behavior, as well as checks for other preserved attributes. The targeted
regression suite completed with 83 tests passed, 16 skipped, and no failures or errors. The original implementation
fails the three TIMES-enabled copy-mode cases, which pass with the fix.

JIRA: https://issues.apache.org/jira/browse/HADOOP-17611
PR: https://github.com/apache/hadoop/pull/8858

Feedback and reviews would be appreciated.

Thanks,
Xinyu Tan

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Re: [Review Request] HADOOP-17611: Preserve DistCp timestamps after chunk concatenation

Posted by Ayush Saxena <ay...@gmail.com>.
Hi Xinyu,
Thanks for reaching out! I'll try to review this over the coming
weekend or the following one if nobody else gets to it before then.

-Ayush

On Fri, 23 Oct 2026 at 14:25, Xinyu Tan <ta...@apache.org> wrote:
>
> Hi Hadoop developers,
>
> I'd like to request a review of a fix for HADOOP-17611, where DistCp's parallel file copy (-blocksperchunk) breaks
> modification time preservation.
>
> When -pt is specified, the mapper preserves timestamps on the chunks, but the subsequent concat operation changes the
> final file's modification time. This patch restores the source modification and access times after concatenation and
> checksum validation, only when TIMES preservation is requested.
>
> The PR adds 14 HDFS contract test cases covering timestamp preservation enabled/disabled, ordinary/direct/atomic
> copies, directory scope, and update/skip behavior, as well as checks for other preserved attributes. The targeted
> regression suite completed with 83 tests passed, 16 skipped, and no failures or errors. The original implementation
> fails the three TIMES-enabled copy-mode cases, which pass with the fix.
>
> JIRA: https://issues.apache.org/jira/browse/HADOOP-17611
> PR: https://github.com/apache/hadoop/pull/8858
>
> Feedback and reviews would be appreciated.
>
> Thanks,
> Xinyu Tan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org