You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2007/04/10 23:07:32 UTC
[jira] Created: (HADOOP-1242) dfs upgrade/downgrade problems
dfs upgrade/downgrade problems
------------------------------
Key: HADOOP-1242
URL: https://issues.apache.org/jira/browse/HADOOP-1242
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.13.0
Reporter: Owen O'Malley
Assigned To: dhruba borthakur
Priority: Critical
Fix For: 0.13.0
I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
:170)
at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
.java:264)
at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
ava:83)
at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-1242:
----------------------------------------
Attachment: failPreUpgradeVersion.patch
This patch leaves old storage (for data-nodes) and old image (for name-nodes) files in the respective old directories after conversions in order to let old versions of Hadoop fail.
These files contain a message explaining their purpose, and particularly saying that they can be safely removed as long as the old versions are not supposed to be used.
I do not enforce creation of the old files if HDFS is created from scratch assuming it is not likely old Hadoops will run in those directories.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-1242:
--------------------------------
Status: Open (was: Patch Available)
A new approach to fixing this issue is being developed
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1242:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Konstantin.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494532 ]
Owen O'Malley commented on HADOOP-1242:
---------------------------------------
The manual steps were to delete a file out of each datanode's local directory. It is not an acceptable solution.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496371 ]
Konstantin Shvachko commented on HADOOP-1242:
---------------------------------------------
I think we should target a more general task here (if any), which could be called "backward incompatibility" I guess.
Namely, a conversion from a pre-upgrade layout versions to the current one should be performed in a way that
any attempt to run old version hdfs software in converted repository would fail.
- For data-nodes this can be achieved by retaining the storage file in its original location but updating its
version to a newer one. That way the old data-node code will complain about not being able to read
future version storage.
- For the name-node we can also write the new version into the old image file or corrupt the image in any other way,
say by placing a message inside "This image is corrupted intentionally. Please do not remove."
The only drawback of this approach I can see is that the old storage and image files will have to stay
in the repositories forever, and even if you create a hdfs from scratch old files will still need to be created
if we want to support backward incompatibility.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by Nigel Daley <nd...@yahoo-inc.com>.
> from the Jira description :
> bq. I ran my test cluster on 0.13 and then tried to run it under
> 0.12. When I downgraded, the namenode would not come up and the
> message said I needed to format the filesystem. I ignored that and
> tried to restart on 0.13, now the datanode will not come up with:
>
> Just for clarification: the patch still does not fix the above
> case, right?
Right. It fixes the issue of going from pre-0.13 to 0.13 back to
pre-0.13. If you start with 0.13, go to pre-0.13 and then back to
0.13 again, you get InconsistentFSStateException.
>> dfs upgrade/downgrade problems
>> ------------------------------
>>
>> Key: HADOOP-1242
>> URL: https://issues.apache.org/jira/browse/
>> HADOOP-1242
>> Project: Hadoop
>> Issue Type: Bug
>> Components: dfs
>> Affects Versions: 0.13.0
>> Reporter: Owen O'Malley
>> Assigned To: Konstantin Shvachko
>> Priority: Blocker
>> Fix For: 0.13.0
>>
>> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch
>>
>>
>> I ran my test cluster on 0.13 and then tried to run it under 0.12.
>> When I downgraded, the namenode would not come up and the message
>> said I needed to format the filesystem. I ignored that and tried
>> to restart on 0.13, now the datanode will not come up with:
>> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode:
>> org.apache.hadoop.
>> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
>> ata is in an inconsistent state: Old layout block directory /local/
>> owen/hadoop/dfs/data/data is missing
>> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded
>> (DataStorage.java
>> :170)
>> at org.apache.hadoop.dfs.Storage
>> $StorageDirectory.analyzeStorage(Storage
>> .java:264)
>> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead
>> (DataStorage.j
>> ava:83)
>> at org.apache.hadoop.dfs.DataNode.startDataNode
>> (DataNode.java:230)
>> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
>> at org.apache.hadoop.dfs.DataNode.makeInstance
>> (DataNode.java:1175)
>> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
>> at org.apache.hadoop.dfs.DataNode.createDataNode
>> (DataNode.java:1140)
>> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498423 ]
Raghu Angadi commented on HADOOP-1242:
--------------------------------------
from the Jira description :
bq. I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
Just for clarification: the patch still does not fix the above case, right?
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498381 ]
Nigel Daley commented on HADOOP-1242:
-------------------------------------
+1 from testing. I've run this patch through the manual test cases attached to HADOOP-702. My only comment is that the descriptive text added to the storage and fsimage files could be clearer.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1242:
----------------------------------
Priority: Blocker (was: Critical)
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1242:
----------------------------------
Attachment: clean-upgrade.patch
This patch just reverses the checks for data and storage. Because the data directory is created after the datanode connects to the namenode it is a better indicator that the data directory is "old".
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1242:
----------------------------------
Status: Patch Available (was: Open)
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-1242:
--------------------------------
Attachment: failPreUpgradeVersion2.patch
Reworded storage and fsimage file messages.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498425 ]
Owen O'Malley commented on HADOOP-1242:
---------------------------------------
I don't think this addresses the problem adequately. I think that you need to create the "corrupt" storage file in all cases, not just upgrade. Otherwise there are too many cases where the user can still hose their cluster. The cases are:
1. a dfs cluster created by 13
2. a dfs cluster updated before this patch goes in
In either case, if the user runs 12 on the cluster, they will need to manually fix their cluster. It would be both easy and safe to _always_ create the corrupt storage if it does not exist.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498421 ]
Raghu Angadi commented on HADOOP-1242:
--------------------------------------
Nigel, it is not clear to me if the manual tests include a test that tests this patch. Could you confirm? thanks.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499889 ]
Hadoop QA commented on HADOOP-1242:
-----------------------------------
-1, build or testing failed
2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12358283/failPreUpgradeVersion3.patch against trunk revision r542595.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/210/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/210/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498427 ]
Raghu Angadi commented on HADOOP-1242:
--------------------------------------
+1 Code looks good.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-1242:
----------------------------------------
Attachment: failPreUpgradeVersion3.patch
Another patch, which enforces creating "corrupted" old storage and image files in all cases.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-1242:
-------------------------------------
Assignee: Owen O'Malley (was: dhruba borthakur)
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494524 ]
dhruba borthakur commented on HADOOP-1242:
------------------------------------------
>From my discussion with Konstantin, i understand that there is a manual sequence of steps to recover from this scenario. Would appreciate it if somebody can describe (in this issue commentary) what these manual steps are.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499960 ]
Owen O'Malley commented on HADOOP-1242:
---------------------------------------
+1
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley reassigned HADOOP-1242:
-----------------------------------
Assignee: Konstantin Shvachko (was: Owen O'Malley)
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494831 ]
Konstantin Shvachko commented on HADOOP-1242:
---------------------------------------------
-1 It is not an acceptable solution.
# By design the storage file is an indicator that the old layout is present.
The order is important since the recovery is based on that order.
# I think this works as designed. You made a mistake, the software detected
inconsistency and warned you. The purpose of the upgrade is to prevent data
loss resulting from software errors or human mistakes.
Trying to ignore inconsistencies is dangerous.
Trying to correct inconsistencies automatically is dangerous too.
# The recovery for this layout version is manual by design. See e.g.
https://issues.apache.org/jira/browse/HADOOP-702#action_12482953
# I do not understand what is expected here. I do not understand what is it blocking.
I think it is a distraction.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499902 ]
Hadoop QA commented on HADOOP-1242:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12358283/failPreUpgradeVersion3.patch applied and successfully tested against trunk revision r542595.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/211/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/211/console
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-1242:
----------------------------------------
Status: Patch Available (was: Open)
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500680 ]
Hadoop QA commented on HADOOP-1242:
-----------------------------------
Integrated in Hadoop-Nightly #107 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/107/)
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494611 ]
Hadoop QA commented on HADOOP-1242:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12357003/clean-upgrade.patch applied and successfully tested against trunk revision r536583.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/127/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/127/console
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499282 ]
Raghu Angadi commented on HADOOP-1242:
--------------------------------------
+1 for the code. Couple of notes:
# Latest patch requires a "image" directory and "image/fsimage" at the name node.
# The new files are not really "CORRUPT", they are in a specific format understood only by post Hadoop-12. Random data in those files will cause 13 to fail.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch, failPreUpgradeVersion.patch, failPreUpgradeVersion2.patch, failPreUpgradeVersion3.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495159 ]
Owen O'Malley commented on HADOOP-1242:
---------------------------------------
> 1. By design the storage file is an indicator that the old layout is present.
> The order is important since the recovery is based on that order.
The "storage" file is a bad indicator precisely because the pre-13 versions of Hadoop's datanodes create it without question. The "data" directory is a much much better indicator, because it is _not_ created automatically. This is far too common a case to allow Hadoop's automatic upgrade to corrupt you data node's directory.
> I think this works as designed.
It may work the way that you intended it, but it is really bad from a usability standpoint. My change isn't perfect, but it handles it much better and you haven't provided any use cases where it is worse.
> You made a mistake, the software detected inconsistency and warned you.
Which is fine, except it also corrupted the repository such that I had to make hand edits to each node in the cluster to fix the problem. That is not ok.
> I do not understand what is expected here. I do not understand what is it blocking.
What is expected is that if you try to bring up a version 12 data node on a version 13 data node directory it will fail. However, when you fix the problem and use version 13 again, it must come up without a problem. Making the administrator log into every single node to delete a file is not ok.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1242) dfs upgrade/downgrade problems
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495215 ]
Konstantin Shvachko commented on HADOOP-1242:
---------------------------------------------
You leave data-node directories in an _inconsistent_ state, namely there are two repositories coexisting together.
This is going to create a confusion on whether the conversion really took place and to what extent.
Another problem is that exactly the same thing can happen if conversion fails after renaming the data directory
to current, but before moving the storage file into the current dir.
With you patch this failure will not be detected.
> dfs upgrade/downgrade problems
> ------------------------------
>
> Key: HADOOP-1242
> URL: https://issues.apache.org/jira/browse/HADOOP-1242
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.13.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Priority: Blocker
> Fix For: 0.13.0
>
> Attachments: clean-upgrade.patch
>
>
> I ran my test cluster on 0.13 and then tried to run it under 0.12. When I downgraded, the namenode would not come up and the message said I needed to format the filesystem. I ignored that and tried to restart on 0.13, now the datanode will not come up with:
> 2007-04-10 11:25:37,448 ERROR org.apache.hadoop.dfs.DataNode: org.apache.hadoop.
> dfs.InconsistentFSStateException: Directory /local/owen/hadoop/dfs/d
> ata is in an inconsistent state: Old layout block directory /local/owen/hadoop/dfs/data/data is missing
> at org.apache.hadoop.dfs.DataStorage.isConversionNeeded(DataStorage.java
> :170)
> at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage
> .java:264)
> at org.apache.hadoop.dfs.DataStorage.recoverTransitionRead(DataStorage.j
> ava:83)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:230)
> at org.apache.hadoop.dfs.DataNode.<init>(DataNode.java:199)
> at org.apache.hadoop.dfs.DataNode.makeInstance(DataNode.java:1175)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1119)
> at org.apache.hadoop.dfs.DataNode.createDataNode(DataNode.java:1140)
> at org.apache.hadoop.dfs.DataNode.main(DataNode.java:1299)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.