You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "vihangk1 (via GitHub)" <gi...@apache.org> on 2023/04/10 17:35:04 UTC
[GitHub] [hive] vihangk1 opened a new pull request, #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
vihangk1 opened a new pull request, #3561:
URL: https://github.com/apache/hive/pull/3561
…tition is deleted
### What changes were proposed in this pull request?
Backports HIVE-26127 to branch-3 from master.
### Why are the changes needed?
The issue reported in HIVE-26127 also affects branch-3.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added a q test from the original patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by "vihangk1 (via GitHub)" <gi...@apache.org>.
vihangk1 commented on PR #3561:
URL: https://github.com/apache/hive/pull/3561#issuecomment-1502096693
Unfortunately, I missed the notification of PR being approved and it was marked stale. Let me reopen this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] closed pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
URL: https://github.com/apache/hive/pull/3561
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ayushtkn commented on a diff in pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on code in PR #3561:
URL: https://github.com/apache/hive/pull/3561#discussion_r961928935
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -4293,7 +4293,7 @@ private void deleteOldPathForReplace(Path destPath, Path oldPath, HiveConf conf,
// But not sure why we changed not to delete the oldPath in HIVE-8750 if it is
// not the destf or its subdir?
isOldPathUnderDestf = isSubDir(oldPath, destPath, oldFs, destFs, false);
- if (isOldPathUnderDestf) {
+ if (isOldPathUnderDestf && oldFs.exists(oldPath)) {
cleanUpOneDirectoryForReplace(oldPath, oldFs, pathFilter, conf, purge, isNeedRecycle);
Review Comment:
Well I see it is a backport but about the fix, having an exist check increased on RPC for every genuine case as well, instead shouldn't we have ignored the FNF in `` cleanUpOneDirectoryForReplace``?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #3561:
URL: https://github.com/apache/hive/pull/3561#issuecomment-1306412705
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by "vihangk1 (via GitHub)" <gi...@apache.org>.
vihangk1 commented on PR #3561:
URL: https://github.com/apache/hive/pull/3561#issuecomment-1696734390
oh, looks like I should have first reopened the PR and then force pushed my rebased branch. Now github is not allowing me to reopen the PR. I will open a new PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3561:
URL: https://github.com/apache/hive/pull/3561#issuecomment-1608522680
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by "vihangk1 (via GitHub)" <gi...@apache.org>.
vihangk1 commented on PR #3561:
URL: https://github.com/apache/hive/pull/3561#issuecomment-1696730989
I would like to reopen this and rebase my PR to latest branch-3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on a diff in pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by GitBox <gi...@apache.org>.
vihangk1 commented on code in PR #3561:
URL: https://github.com/apache/hive/pull/3561#discussion_r962171621
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -4293,7 +4293,7 @@ private void deleteOldPathForReplace(Path destPath, Path oldPath, HiveConf conf,
// But not sure why we changed not to delete the oldPath in HIVE-8750 if it is
// not the destf or its subdir?
isOldPathUnderDestf = isSubDir(oldPath, destPath, oldFs, destFs, false);
- if (isOldPathUnderDestf) {
+ if (isOldPathUnderDestf && oldFs.exists(oldPath)) {
cleanUpOneDirectoryForReplace(oldPath, oldFs, pathFilter, conf, purge, isNeedRecycle);
Review Comment:
Your comment makes sense to me. I think that is a valid way to handle this as well. But since this is a backport, if we want to do this we should do it in a separate PR in master branch first to avoid git conflicts in future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on a diff in pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by GitBox <gi...@apache.org>.
vihangk1 commented on code in PR #3561:
URL: https://github.com/apache/hive/pull/3561#discussion_r966258042
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -4293,7 +4293,7 @@ private void deleteOldPathForReplace(Path destPath, Path oldPath, HiveConf conf,
// But not sure why we changed not to delete the oldPath in HIVE-8750 if it is
// not the destf or its subdir?
isOldPathUnderDestf = isSubDir(oldPath, destPath, oldFs, destFs, false);
- if (isOldPathUnderDestf) {
+ if (isOldPathUnderDestf && oldFs.exists(oldPath)) {
cleanUpOneDirectoryForReplace(oldPath, oldFs, pathFilter, conf, purge, isNeedRecycle);
Review Comment:
@ayushtkn Will you be able to approve this PR since you have already looked into this? I can create a follow-up PR which incorporates you feedback on master branch if you want.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by "vihangk1 (via GitHub)" <gi...@apache.org>.
vihangk1 commented on PR #3561:
URL: https://github.com/apache/hive/pull/3561#issuecomment-1696731310
reopening this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] closed pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #3561: HIVE-26127: Insert overwrite throws FileNotFound when destination par…
URL: https://github.com/apache/hive/pull/3561
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org