You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Jiyoung Yoo (Code Review)" <ge...@cloudera.org> on 2026/07/15 18:28:41 UTC

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Jiyoung Yoo has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24588


Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................

IMPALA-15117: Optimize getFirstLevelAcidDirPath

In the original implementation of AcidUtils.getFirstLevelAcidDirPath(),
the FileSystem.isDirectory() API was called recursively for each path
component. This introduces unnecessary RPC overhead during table loading.

This change refactors the logic to leverage Path string operations by
traversing upwards from the file's parent directory to detect the ACID
directory structure

1. Eliminated FileSystem.isDirectory() RPCs:
   Since the input filePath is guaranteed to be a valid data file
   during metadata load, we can assume that all of its parent paths
   are directories

2. Preserved backward compatibility:
   The original recursive behavior handled nested subdirectories
   inside ACID folders. To maintain the same behavior, this change
   preserved the exact same path-merging logic (e.g., returning
   "delta_000001_000001_0000/subdir" for a file path like
   “partition_dir/delta_000001_000001_0000/sub_dir/filename")

Testing: AcidUtilsTest.testGetFirstLevelAcidDirPath

Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/AcidUtils.java
M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java
3 files changed, 77 insertions(+), 36 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/24588/2
-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 2
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................

IMPALA-15117: Optimize getFirstLevelAcidDirPath

In the original implementation of AcidUtils.getFirstLevelAcidDirPath(),
the FileSystem.isDirectory() API was called recursively for each path
component. This introduced unnecessary RPC overhead during table loading.

This change refactors the logic to leverage Path string operations by
traversing upwards from the file's parent directory to detect the ACID
directory structure.

1. Eliminated FileSystem.isDirectory() RPCs:
   Since the input filePath is guaranteed to be a valid data file
   during metadata load, we can assume that all of its parent paths
   are directories

2. Preserved backward compatibility:
   The original recursive behavior handled nested subdirectories
   inside ACID folders. To maintain the same behavior, this change
   preserves the exact same path-merging logic (e.g., returning
   "delta_000001_000001_0000/sub_dir" for a file path like
   "partition_dir/delta_000001_000001_0000/sub_dir/filename")

Testing:
- Passed core tests.
- Added AcidUtilsTest.testGetFirstLevelAcidDirPath.

Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Reviewed-on: http://gerrit.cloudera.org:8080/24588
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/AcidUtils.java
M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java
3 files changed, 81 insertions(+), 38 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 7
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Jiyoung Yoo (Code Review)" <ge...@cloudera.org>.
Jiyoung Yoo has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@24
PS2, Line 24: import java.io.IOException;
> These aren't caused by your patch but would be nice to cleanup. We have a n
Done


http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@38
PS2, Line 38: import org.apache.hadoop.fs.FileSystem;
> unused import: org.apache.hadoop.fs.FileSystem
Done


http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@214
PS2, Line 214:         return acidRoot.getName() + parentStr.substring(rootStr.length());
> Can you add a comment describing this branch? I don't understand why it's n
You are right, we don't need to compare parentStr and rootStr because they always shares the same prefix. I added some comments with the fix. Thanks!



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 2
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 21:34:58 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Thu, 16 Jul 2026 23:52:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/23256/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 22:04:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@214
PS2, Line 214:       //    -> Returns base_0000005/0000
> You are right, we don't need to compare parentStr and rootStr because they 
Ack



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 22:21:21 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@214
PS2, Line 214:         return acidRoot.getName() + parentStr.substring(rootStr.length());
Can you add a comment describing this branch? I don't understand why it's needed.



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 2
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 18:39:33 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 6
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 18:47:14 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 5
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 18:46:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 6
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 23:12:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 5: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 5
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 18:46:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Jiyoung Yoo (Code Review)" <ge...@cloudera.org>.
Hello Yida Wu, Sai Hemanth Gantasala, Michael Smith, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/24588

to look at the new patch set (#5).

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................

IMPALA-15117: Optimize getFirstLevelAcidDirPath

In the original implementation of AcidUtils.getFirstLevelAcidDirPath(),
the FileSystem.isDirectory() API was called recursively for each path
component. This introduced unnecessary RPC overhead during table loading.

This change refactors the logic to leverage Path string operations by
traversing upwards from the file's parent directory to detect the ACID
directory structure.

1. Eliminated FileSystem.isDirectory() RPCs:
   Since the input filePath is guaranteed to be a valid data file
   during metadata load, we can assume that all of its parent paths
   are directories

2. Preserved backward compatibility:
   The original recursive behavior handled nested subdirectories
   inside ACID folders. To maintain the same behavior, this change
   preserves the exact same path-merging logic (e.g., returning
   "delta_000001_000001_0000/sub_dir" for a file path like
   "partition_dir/delta_000001_000001_0000/sub_dir/filename")

Testing:
- Passed core tests.
- Added AcidUtilsTest.testGetFirstLevelAcidDirPath.

Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/AcidUtils.java
M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java
3 files changed, 81 insertions(+), 38 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/24588/5
-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 5
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13765/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 6
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 18:47:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 5: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 5
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 21:31:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 2:

(2 comments)

gerrit-auto-critic failed. You can reproduce it locally using command:

  python3 bin/jenkins/critique-gerrit-review.py --dryrun

To run it, you might need a virtual env with Python3's venv installed.

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@24
PS2, Line 24: import java.io.IOException;
unused import: java.io.IOException


http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@38
PS2, Line 38: import org.apache.hadoop.fs.FileSystem;
unused import: org.apache.hadoop.fs.FileSystem



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 2
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 18:30:58 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Jiyoung Yoo (Code Review)" <ge...@cloudera.org>.
Hello Yida Wu, Sai Hemanth Gantasala, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/24588

to look at the new patch set (#3).

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................

IMPALA-15117: Optimize getFirstLevelAcidDirPath

In the original implementation of AcidUtils.getFirstLevelAcidDirPath(),
the FileSystem.isDirectory() API was called recursively for each path
component. This introduces unnecessary RPC overhead during table loading.

This change refactors the logic to leverage Path string operations by
traversing upwards from the file's parent directory to detect the ACID
directory structure

1. Eliminated FileSystem.isDirectory() RPCs:
   Since the input filePath is guaranteed to be a valid data file
   during metadata load, we can assume that all of its parent paths
   are directories

2. Preserved backward compatibility:
   The original recursive behavior handled nested subdirectories
   inside ACID folders. To maintain the same behavior, this change
   preserved the exact same path-merging logic (e.g., returning
   "delta_000001_000001_0000/subdir" for a file path like
   “partition_dir/delta_000001_000001_0000/sub_dir/filename")

Testing: AcidUtilsTest.testGetFirstLevelAcidDirPath

Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/AcidUtils.java
M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java
3 files changed, 79 insertions(+), 38 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/24588/3
-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 3
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/23255/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 3
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 21:55:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13754/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Thu, 16 Jul 2026 19:28:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 2:

(1 comment)

Thanks for the patch! I've added a few others to help review.

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/2/fe/src/main/java/org/apache/impala/util/AcidUtils.java@24
PS2, Line 24: import java.io.IOException;
> unused import: java.io.IOException
These aren't caused by your patch but would be nice to cleanup. We have a new job to detect unused imports that only flags files when they're changed.



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 2
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 18:36:15 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Jiyoung Yoo (Code Review)" <ge...@cloudera.org>.
Jiyoung Yoo has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/24588/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/24588/4//COMMIT_MSG@29
PS4, Line 29: Testing: AcidUtilsTest.testGetFirstLevelAcidDirPath
> Did this run against the core tests? If yes, we can mention here in the "Te
Done


http://gerrit.cloudera.org:8080/#/c/24588/4/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/4/fe/src/main/java/org/apache/impala/util/AcidUtils.java@180
PS4, Line 180: but has been refactored to
             :    * eliminate costly HDFS PRC calls(e.g. FileSystem.isDirectory)
> It is mentioned in the commit message, but it's better to put it directly i
Done


http://gerrit.cloudera.org:8080/#/c/24588/4/fe/src/main/java/org/apache/impala/util/AcidUtils.java@181
PS4, Line 181: PRC
> nit. RPC
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 16:56:57 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13763/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 5
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 17:01:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Yida Wu (Code Review)" <ge...@cloudera.org>.
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/24588/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/24588/4//COMMIT_MSG@29
PS4, Line 29: Testing: AcidUtilsTest.testGetFirstLevelAcidDirPath
Did this run against the core tests? If yes, we can mention here in the "Testing" section


http://gerrit.cloudera.org:8080/#/c/24588/4/fe/src/main/java/org/apache/impala/util/AcidUtils.java
File fe/src/main/java/org/apache/impala/util/AcidUtils.java:

http://gerrit.cloudera.org:8080/#/c/24588/4/fe/src/main/java/org/apache/impala/util/AcidUtils.java@180
PS4, Line 180: but has been refactored to
             :    * eliminate costly HDFS PRC calls(e.g. FileSystem.isDirectory)
It is mentioned in the commit message, but it's better to put it directly in the Javadoc to explain why we can safely remove the isDirectory call.
"Since the input filePath is guaranteed to be a valid data file during metadata load"


http://gerrit.cloudera.org:8080/#/c/24588/4/fe/src/main/java/org/apache/impala/util/AcidUtils.java@181
PS4, Line 181: PRC
nit. RPC



-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Sat, 18 Jul 2026 03:46:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Yida Wu (Code Review)" <ge...@cloudera.org>.
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 4: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Sat, 18 Jul 2026 03:46:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/23319/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 5
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Jul 2026 17:20:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24588 )

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/23251/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 2
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 15 Jul 2026 18:50:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-15117: Optimize getFirstLevelAcidDirPath

Posted by "Jiyoung Yoo (Code Review)" <ge...@cloudera.org>.
Hello Yida Wu, Sai Hemanth Gantasala, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/24588

to look at the new patch set (#4).

Change subject: IMPALA-15117: Optimize getFirstLevelAcidDirPath
......................................................................

IMPALA-15117: Optimize getFirstLevelAcidDirPath

In the original implementation of AcidUtils.getFirstLevelAcidDirPath(),
the FileSystem.isDirectory() API was called recursively for each path
component. This introduces unnecessary RPC overhead during table loading.

This change refactors the logic to leverage Path string operations by
traversing upwards from the file's parent directory to detect the ACID
directory structure

1. Eliminated FileSystem.isDirectory() RPCs:
   Since the input filePath is guaranteed to be a valid data file
   during metadata load, we can assume that all of its parent paths
   are directories

2. Preserved backward compatibility:
   The original recursive behavior handled nested subdirectories
   inside ACID folders. To maintain the same behavior, this change
   preserved the exact same path-merging logic (e.g., returning
   "delta_000001_000001_0000/subdir" for a file path like
   “partition_dir/delta_000001_000001_0000/sub_dir/filename")

Testing: AcidUtilsTest.testGetFirstLevelAcidDirPath

Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/AcidUtils.java
M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java
3 files changed, 79 insertions(+), 38 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/24588/4
-- 
To view, visit http://gerrit.cloudera.org:8080/24588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I390183039181bb62d6818c8e869738ddf01c2f6b
Gerrit-Change-Number: 24588
Gerrit-PatchSet: 4
Gerrit-Owner: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Jiyoung Yoo <ji...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>