You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/07/29 06:43:54 UTC
[GitHub] [hive] Neilxzn opened a new pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Neilxzn opened a new pull request #2544:
URL: https://github.com/apache/hive/pull/2544
### What changes were proposed in this pull request?
obtaine delegation tokens for hive scratchDirs before hive submit mapreduce job.
### Why are the changes needed?
https://issues.apache.org/jira/browse/HIVE-25401
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
no test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #2544:
URL: https://github.com/apache/hive/pull/2544#issuecomment-974730527
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] Neilxzn commented on pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
Neilxzn commented on pull request #2544:
URL: https://github.com/apache/hive/pull/2544#issuecomment-888852278
cc @sunchao . Thank you for your reivew.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #2544:
URL: https://github.com/apache/hive/pull/2544#issuecomment-1018984793
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #2544:
URL: https://github.com/apache/hive/pull/2544#issuecomment-974730527
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] Neilxzn commented on a change in pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
Neilxzn commented on a change in pull request #2544:
URL: https://github.com/apache/hive/pull/2544#discussion_r754177729
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
##########
@@ -4990,4 +4991,19 @@ public static boolean arePathsEqualOrWithin(Path p1, Path p2) {
return ((p1.toString().toLowerCase().indexOf(p2.toString().toLowerCase()) > -1) ||
(p2.toString().toLowerCase().indexOf(p1.toString().toLowerCase()) > -1)) ? true : false;
}
+
+ /**
+ * Convenience method to obtain delegation tokens
+ * corresponding to the paths passed for mapReduce job.
+ * @param job jonconf
+ * @param ps array of paths
+ */
+ public static void setToken(JobConf job, Path[] ps) {
+ try {
+ TokenCache.obtainTokensForNamenodes(job.getCredentials(),
+ ps, job);
+ } catch (IOException ex) {
+ LOG.error("Error in setToken ", ex);
Review comment:
Thank you for your review!
Agree with you and I have removed the `try catch`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] closed pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #2544:
URL: https://github.com/apache/hive/pull/2544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] Neilxzn commented on pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
Neilxzn commented on pull request #2544:
URL: https://github.com/apache/hive/pull/2544#issuecomment-902388763
The fail test `testGetPartitionWithAuthInfoNoDbName ` is not related to this patch.
@sankarh Can you help me review it? Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kgyrtkirk commented on a change in pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
kgyrtkirk commented on a change in pull request #2544:
URL: https://github.com/apache/hive/pull/2544#discussion_r754101348
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
##########
@@ -4990,4 +4991,19 @@ public static boolean arePathsEqualOrWithin(Path p1, Path p2) {
return ((p1.toString().toLowerCase().indexOf(p2.toString().toLowerCase()) > -1) ||
(p2.toString().toLowerCase().indexOf(p1.toString().toLowerCase()) > -1)) ? true : false;
}
+
+ /**
+ * Convenience method to obtain delegation tokens
+ * corresponding to the paths passed for mapReduce job.
+ * @param job jonconf
+ * @param ps array of paths
+ */
+ public static void setToken(JobConf job, Path[] ps) {
+ try {
+ TokenCache.obtainTokensForNamenodes(job.getCredentials(),
+ ps, job);
+ } catch (IOException ex) {
+ LOG.error("Error in setToken ", ex);
Review comment:
I don't think any errors should be ignored here - if it will not be able to obtain the token; it will not work; or that's not the case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] Neilxzn commented on pull request #2544: HIVE-25401: Insert overwrite a table which location is on other cluster fail in kerberos cluster
Posted by GitBox <gi...@apache.org>.
Neilxzn commented on pull request #2544:
URL: https://github.com/apache/hive/pull/2544#issuecomment-890748725
I think the failed test is not related to this patch. Please run test again. Thank you. @kgyrtkirk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org