You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sergio Peña (JIRA)" <ji...@apache.org> on 2014/12/12 21:58:13 UTC
[jira] [Commented] (HIVE-9087) The move task does not handle
properly in the case of loading data from the local file system path.
[ https://issues.apache.org/jira/browse/HIVE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244768#comment-14244768 ]
Sergio Peña commented on HIVE-9087:
-----------------------------------
Hi Ferdinand,
What if we validate that the scheme is not 'hdfs' instead of 'file'? As what I understood is that this breaks only for non-hdfs files, but Hive tests run on another different schema 'pfile' as well; so this might fail in the future.
The other thing is to save the value of srcf.toUri() to an URI object, and then get validate the scheme from this URI object. This to avoid calling toUri() twice, and make the code clean with shorter lines.
> The move task does not handle properly in the case of loading data from the local file system path.
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-9087
> URL: https://issues.apache.org/jira/browse/HIVE-9087
> Project: Hive
> Issue Type: Sub-task
> Reporter: Ferdinand Xu
> Assignee: Ferdinand Xu
> Attachments: HIVE-9087.patch
>
>
> NO PRECOMMIT TESTS
> The following exception will be thrown.
> load data local inpath "/root/testdata/encrypt_data.txt" overwrite into table unencrypteddb.src;
> Getting log thread is interrupted, since query is done!
> 14/12/12 19:17:12 ERROR exec.Task: Failed with exception Wrong FS: file:/root/testdata/encrypt_data.txt, expected: hdfs://localhost:9000
> java.lang.IllegalArgumentException: Wrong FS: file:/root/testdata/encrypt_data.txt, expected: hdfs://localhost:9000
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
> at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:465)
> at org.apache.hadoop.hive.common.FileUtils.isSubDir(FileUtils.java:616)
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2340)
> at org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:2659)
> at org.apache.hadoop.hive.ql.metadata.Table.replaceFiles(Table.java:666)
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1571)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:289)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1644)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1217)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1039)
> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144)
> at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69)
> at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
> at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> 14/12/12 19:17:12 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> 14/12/12 19:17:12 ERROR operation.Operation: Error running hive query:
> org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:314)
> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:146)
> at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69)
> at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
> at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask (state=08S01,code=1)
> java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:296)
> at org.apache.hive.beeline.Commands.execute(Commands.java:781)
> at org.apache.hive.beeline.Commands.sql(Commands.java:665)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:914)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:776)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:738)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:452)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)