You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (JIRA)" <ji...@apache.org> on 2019/08/07 13:13:00 UTC
[jira] [Commented] (IMPALA-8841) Dataload flakiness:
hive-exec-*.jar changed on src filesystem
[ https://issues.apache.org/jira/browse/IMPALA-8841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902053#comment-16902053 ]
Csaba Ringhofer commented on IMPALA-8841:
-----------------------------------------
Our theory is that concurrent queries that start Tez sessions try to copy the .jar at the same time, leading some of them to see different mtime than expected.
If the copy is done only during the very first sessions when the file doesn't exist yet, then it should be enough to run a query (that needs Tez) before starting parallel work, or to simply copy the .jar to its destination.
> Dataload flakiness: hive-exec-*.jar changed on src filesystem
> -------------------------------------------------------------
>
> Key: IMPALA-8841
> URL: https://issues.apache.org/jira/browse/IMPALA-8841
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.3.0
> Reporter: Csaba Ringhofer
> Priority: Major
>
> The following exception appears time to time in Hive 3 +Tez builds:
> {code}
> Failing this attempt.Diagnostics: [2019-07-25 11:26:46.931]Resource hdfs://localhost:20500/user/ubuntu/.hiveJars/hive-exec-3.1.0.7.0.0.0-280-f198e31861337336ae45a4c9779fc5db5af5d097a4b2000f8a96876573d03c8e.jar changed on src filesystem (expected 1564054001302, was 1564054001832
> java.io.IOException: Resource hdfs://localhost:20500/user/ubuntu/.hiveJars/hive-exec-3.1.0.7.0.0.0-280-f198e31861337336ae45a4c9779fc5db5af5d097a4b2000f8a96876573d03c8e.jar changed on src filesystem (expected 1564054001302, was 1564054001832
> at org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:273)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:67)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:414)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:411)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:411)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:242)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:235)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:223)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org