You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2014/10/28 07:17:34 UTC
[jira] [Comment Edited] (HIVE-7623) hive partition rename fails if
filesystem cache is disabled
[ https://issues.apache.org/jira/browse/HIVE-7623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186442#comment-14186442 ]
Thejas M Nair edited comment on HIVE-7623 at 10/28/14 6:17 AM:
---------------------------------------------------------------
[~hagleitn] I agree, makes sense to include it in 0.14 . It is a small change, should be safe.
But this is fix is not as crucial as before, since we fixed the issue in HS2 unsecure mode, that was causing FileSystem cache leaks.
cc [~vikram.dixit]
was (Author: thejas):
[~hagleitn] I agree, makes sense to include it in 0.14 . It is a small change, should be safe.
But this is fix is not as crucial as before, since we fixed the issue in HS2 unsecure mode, that was causing FileSystem cache leaks.
> hive partition rename fails if filesystem cache is disabled
> -----------------------------------------------------------
>
> Key: HIVE-7623
> URL: https://issues.apache.org/jira/browse/HIVE-7623
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.13.0, 0.13.1
> Reporter: agate
> Assignee: Navis
> Fix For: 0.15.0
>
> Attachments: HIVE-7623.1.patch.txt
>
>
> Seems to be similar issue https://issues.apache.org/jira/browse/HIVE-3815 when calling alterPartition (when renaming partitions)
> Setting fs.hdfs.impl.disable.cache=false and fs.file.impl.disable.cache=falseworks around this problem
> Error:
> =====
> 2014-08-05 21:46:14,522 ERROR [pool-3-thread-1]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invoke(143)) - InvalidOperationException(message:table new location hdfs://hadoop-namenode:8020/user/hive/warehouse/sample_logs/XX=AA/YY=123 is on a different file system than the old location hdfs://hadoop-namenode:8020/user/hive/warehouse/sample_logs/XX=AA/YY=456. This operation is not supported)
> at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartition(HiveAlterHandler.java:361)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rename_partition(HiveMetaStore.java:2629)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rename_partition(HiveMetaStore.java:2602)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:622)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at com.sun.proxy.$Proxy5.rename_partition(Unknown Source)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$rename_partition.getResult(ThriftHiveMetastore.java:9057)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$rename_partition.getResult(ThriftHiveMetastore.java:9041)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> Looking at the code apache-hive-0.13.1-src/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java on line 361 see that its using != to compare filesystem objects
> // check that src and dest are on the same file system
> if (srcFs != destFs) {
> throw new InvalidOperationException("table new location " + destPath
> + " is on a different file system than the old location "
> + srcPath + ". This operation is not supported");
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)