You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2007/05/15 21:07:16 UTC
[jira] Created: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
-------------------------------------------------------------------------
Key: HADOOP-1372
URL: https://issues.apache.org/jira/browse/HADOOP-1372
Project: Hadoop
Issue Type: Bug
Components: dfs
Reporter: Devaraj Das
Fix For: 0.14.0
I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
java.io.FileNotFoundException: /export/crawlspace4/kryptonite/ddas/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:179)
at java.io.FileOutputStream.(FileOutputStream.java:131)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506044 ]
Hadoop QA commented on HADOOP-1372:
-----------------------------------
Integrated in Hadoop-Nightly #127 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/127/])
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur reassigned HADOOP-1372:
----------------------------------------
Assignee: dhruba borthakur
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1372:
-------------------------------------
Status: Patch Available (was: Open)
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1372:
-------------------------------------
Attachment: localDirAllocator.patch
DFSClient now uses LocalDirAllocator.
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: localDirAllocator.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500927 ]
Devaraj Das commented on HADOOP-1372:
-------------------------------------
+1
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1372:
-------------------------------------
Attachment: (was: localDirAllocator.patch)
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1372:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Dhruba!
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1372:
----------------------------------
Description:
I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:179)
at java.io.FileOutputStream.(FileOutputStream.java:131)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
was:
I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
java.io.FileNotFoundException: /export/crawlspace4/kryptonite/ddas/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:179)
at java.io.FileOutputStream.(FileOutputStream.java:131)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Fix For: 0.14.0
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500958 ]
Devaraj Das commented on HADOOP-1372:
-------------------------------------
I think this patch needs to be relooked at. Please see my comment on HADOOP-1396 - http://issues.apache.org/jira/browse/HADOOP-1396#action_12500956
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1372:
-------------------------------------
Attachment: localDirAllocator2.patch
Incorporated review comments from Devaraj.
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12504842 ]
Hadoop QA commented on HADOOP-1372:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12359680/localDirAllocator2.patch applied and successfully tested against trunk revision r547159.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/283/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/283/console
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1372) DFS Clients should start using the
org.apache.hadoop.fs.LocalDirAllocator
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12504792 ]
Devaraj Das commented on HADOOP-1372:
-------------------------------------
+1
> DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator
> -------------------------------------------------------------------------
>
> Key: HADOOP-1372
> URL: https://issues.apache.org/jira/browse/HADOOP-1372
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Devaraj Das
> Assignee: dhruba borthakur
> Fix For: 0.14.0
>
> Attachments: localDirAllocator2.patch
>
>
> I encountered this exception during one of the randomwriter runs. I think this situation can be improved by using org.apache.hadoop.fs.LocalDirAllocator that has been written to handle these kind of problems. I set the fix version as 0.14 but wonder whether it makes sense to have it in 0.13 itself (since the amount of code change would not be much).
> java.io.FileNotFoundException: /local/dfs/data/tmp/client-1299146109450372217 (Read-only file system)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:179)
> at java.io.FileOutputStream.(FileOutputStream.java:131)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.endBlock(DFSClient.java:1356)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:1273)
> at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.write(DFSClient.java:1255)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.write(ChecksumFileSystem.java:402)
> at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:38)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:775)
> at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:152)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:187)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1709)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.