You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Chinna Rao Lalam (JIRA)" <ji...@apache.org> on 2011/01/06 15:26:47 UTC
[jira] Created: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Alter command execution "when HDFS is down" results in holding stale data in MetaStore
---------------------------------------------------------------------------------------
Key: HIVE-1897
URL: https://issues.apache.org/jira/browse/HIVE-1897
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.6.0
Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Lets consider, the *"DFS"* is down ,
And on executing an alter query say *"alter table firsttable rename to secondtable"*.
the query execution fails with the following exception:
{color:red}
InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
{color}
Now after starting the *DFS* and then executing the same query , the client gets the following exception:
{color:red}
NoSuchObjectException(message:default.firsttable table not found)
{color}
h4.Root Cause
In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1897:
-----------------------------------
Attachment: HIVE-1897.1.patch
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Attachments: HIVE-1897.1.patch
>
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985818#action_12985818 ]
Namit Jain commented on HIVE-1897:
----------------------------------
This still does not work if the client dies after making the metadata change.
However, this does reduce the window.
+1
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Fix For: 0.7.0
>
> Attachments: HIVE-1897.1.patch
>
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-1897:
-----------------------------------
Fix Version/s: 0.7.0
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Fix For: 0.7.0
>
> Attachments: HIVE-1897.1.patch
>
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain reassigned HIVE-1897:
--------------------------------
Assignee: Chinna Rao Lalam
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Fix For: 0.7.0
>
> Attachments: HIVE-1897.1.patch
>
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1897:
-----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Chinna Rao Lalam
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Fix For: 0.7.0
>
> Attachments: HIVE-1897.1.patch
>
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1897:
-----------------------------------
Release Note: HiveAlterHandler ( alterTable ) , In HDFS operation catch block added renaming the table name to old table name.
Status: Patch Available (was: Open)
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Attachments: HIVE-1897.1.patch
>
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1897) Alter command execution "when HDFS is
down" results in holding stale data in MetaStore
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985666#action_12985666 ]
Chinna Rao Lalam commented on HIVE-1897:
----------------------------------------
In HiveAlterHandler ( alterTable ) , if the HDFS operation failed then its better to rename the table to the old table name. To avoid the inconsistency b/w Metadata and HDFS data.
h4. Old code snippet
h4.
{code}
if (success && moveData) {
// change the file name in hdfs
// check that src exists otherwise there is no need to copy the data
try {
if (srcFs.exists(srcPath)) {
// rename the src to destination
srcFs.rename(srcPath, destPath);
}
} catch (IOException e) {
throw new InvalidOperationException("Unable to access old location "
+ srcPath + " for table " + dbname + "." + name);
}
{code}
h4. Changes Proposed
Before throwing the "InvalidOperationException" , rename the table name to the old table name.
{code}
if (success && moveData) {
// change the file name in hdfs
// check that src exists otherwise there is no need to copy the data
try {
if (srcFs.exists(srcPath)) {
// rename the src to destination
srcFs.rename(srcPath, destPath);
}
} catch (IOException e) {
boolean revertMetaDataTransaction = false;
try
{
msdb.openTransaction();
// Alter the table name to have the old table name.
msdb.alterTable(dbname, newt.getTableName(), oldt);
revertMetaDataTransaction = msdb.commitTransaction();
}
catch (Exception e1)
{
LOG.error("Reverting metadata opeation failed During HDFS operation failed",e1);
if(!revertMetaDataTransaction)
{
msdb.rollbackTransaction();
}
}
throw new InvalidOperationException("Unable to access old location "
+ srcPath + " for table " + dbname + "." + name);
}
}
{code}
> Alter command execution "when HDFS is down" results in holding stale data in MetaStore
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-1897
> URL: https://issues.apache.org/jira/browse/HIVE-1897
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Hive0.6.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
>
> Lets consider, the *"DFS"* is down ,
> And on executing an alter query say *"alter table firsttable rename to secondtable"*.
> the query execution fails with the following exception:
> {color:red}
> InvalidOperationException(message:Unable to access old location hdfs://localhost:9000/user/hive/warehouse/firsttable for table default.firsttable)
> {color}
> Now after starting the *DFS* and then executing the same query , the client gets the following exception:
> {color:red}
> NoSuchObjectException(message:default.firsttable table not found)
> {color}
> h4.Root Cause
> In Alter Query execution flow, first *"MetaStore"* operation is executed successfully and then *"DFS"* operation is started. In this scenario, *"DFS"* is down. As a result, execution of the query failed and partial information of the operation is saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.