You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Benjamin Francisoud (JIRA)" <ji...@apache.org> on 2008/01/09 17:46:34 UTC
[jira] Created: (HADOOP-2561)
/tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
/tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
---------------------------------------------------------------------------
Key: HADOOP-2561
URL: https://issues.apache.org/jira/browse/HADOOP-2561
Project: Hadoop
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Benjamin Francisoud
Diretory "/tmp/hadoop-${user}/dfs/tmp/tmp" is being filled with those kinfd of files: client-226966559287638337420857.tmp
I tried to look at the code and found:
h3. DFSClient.java
src/java/org/apache/hadoop/dfs/DFSClient.java
{code:java}
private void closeBackupStream() throws IOException {...}
/* Similar to closeBackupStream(). Theoritically deleting a file
* twice could result in deleting a file that we should not.
*/
private void deleteBackupFile() {...}
private File newBackupFile() throws IOException {
String name = "tmp" + File.separator +
"client-" + Math.abs(r.nextLong());
File result = dirAllocator.createTmpFileForWrite(name,
2 * blockSize,
conf);
return result;
}
{code}
h3. LocalDirAllocator
src/java/org/apache/hadoop/fs/LocalDirAllocator.java#AllocatorPerContext.java
{code:java}
/** Creates a file on the local FS. Pass size as -1 if not known apriori. We
* round-robin over the set of disks (via the configured dirs) and return
* a file on the first path which has enough space. The file is guaranteed
* to go away when the JVM exits.
*/
public File createTmpFileForWrite(String pathStr, long size,
Configuration conf) throws IOException {
// find an appropriate directory
Path path = getLocalPathForWrite(pathStr, size, conf);
File dir = new File(path.getParent().toUri().getPath());
String prefix = path.getName();
// create a temp file on this directory
File result = File.createTempFile(prefix, null, dir);
result.deleteOnExit();
return result;
}
{code}
First it seems to me it's a bit of a mess here I don't know if it's DFSClient.java#deleteBackupFile() or LocalDirAllocator#createTmpFileForWrite() {deleteOnExit(); ) who is call ... or both. Why not keep it dry and delete it only once.
But the most important is the "deleteOnExit();" since it mean if it is never restarted it will never delete files :(
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2561)
/tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559325#action_12559325 ]
Konstantin Shvachko commented on HADOOP-2561:
---------------------------------------------
Good catch.
It looks like the problem here is more general than what your solution does.
The client writes to the backup file for as long as it reaches the block size. So if the client fails anywhere between 2 individual writes the backupFile will not be deleted. Simple try-catch will not solve it. And there is not much you can do about it.
This issue should probably be resolved as a part of HADOOP-1707, which intends to eliminate the local backup file completely.
Benjamin, could you please take a look at 1707 to make sure the solution works for you.
> /tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
> ---------------------------------------------------------------------------
>
> Key: HADOOP-2561
> URL: https://issues.apache.org/jira/browse/HADOOP-2561
> Project: Hadoop
> Issue Type: Bug
> Components: dfs, fs
> Affects Versions: 0.14.0
> Reporter: Benjamin Francisoud
> Attachments: patch-DFSClient-HADOOP-2561.diff
>
>
> Diretory "/tmp/hadoop-${user}/dfs/tmp/tmp" is being filled with those kinfd of files: client-226966559287638337420857.tmp
> I tried to look at the code and found:
> h3. DFSClient.java
> src/java/org/apache/hadoop/dfs/DFSClient.java
> {code:java}
> private void closeBackupStream() throws IOException {...}
> /* Similar to closeBackupStream(). Theoritically deleting a file
> * twice could result in deleting a file that we should not.
> */
> private void deleteBackupFile() {...}
> private File newBackupFile() throws IOException {
> String name = "tmp" + File.separator +
> "client-" + Math.abs(r.nextLong());
> File result = dirAllocator.createTmpFileForWrite(name,
> 2 * blockSize,
> conf);
> return result;
> }
> {code}
> h3. LocalDirAllocator
> src/java/org/apache/hadoop/fs/LocalDirAllocator.java#AllocatorPerContext.java
> {code:java}
> /** Creates a file on the local FS. Pass size as -1 if not known apriori. We
> * round-robin over the set of disks (via the configured dirs) and return
> * a file on the first path which has enough space. The file is guaranteed
> * to go away when the JVM exits.
> */
> public File createTmpFileForWrite(String pathStr, long size,
> Configuration conf) throws IOException {
> // find an appropriate directory
> Path path = getLocalPathForWrite(pathStr, size, conf);
> File dir = new File(path.getParent().toUri().getPath());
> String prefix = path.getName();
> // create a temp file on this directory
> File result = File.createTempFile(prefix, null, dir);
> result.deleteOnExit();
> return result;
> }
> {code}
> First it seems to me it's a bit of a mess here I don't know if it's DFSClient.java#deleteBackupFile() or LocalDirAllocator#createTmpFileForWrite() {deleteOnExit(); ) who is call ... or both. Why not keep it dry and delete it only once.
> But the most important is the "deleteOnExit();" since it mean if it is never restarted it will never delete files :(
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2561)
/tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
Posted by "Benjamin Francisoud (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Francisoud updated HADOOP-2561:
----------------------------------------
Component/s: fs
dfs
> /tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
> ---------------------------------------------------------------------------
>
> Key: HADOOP-2561
> URL: https://issues.apache.org/jira/browse/HADOOP-2561
> Project: Hadoop
> Issue Type: Bug
> Components: dfs, fs
> Affects Versions: 0.14.0
> Reporter: Benjamin Francisoud
> Attachments: patch-DFSClient-HADOOP-2561.diff
>
>
> Diretory "/tmp/hadoop-${user}/dfs/tmp/tmp" is being filled with those kinfd of files: client-226966559287638337420857.tmp
> I tried to look at the code and found:
> h3. DFSClient.java
> src/java/org/apache/hadoop/dfs/DFSClient.java
> {code:java}
> private void closeBackupStream() throws IOException {...}
> /* Similar to closeBackupStream(). Theoritically deleting a file
> * twice could result in deleting a file that we should not.
> */
> private void deleteBackupFile() {...}
> private File newBackupFile() throws IOException {
> String name = "tmp" + File.separator +
> "client-" + Math.abs(r.nextLong());
> File result = dirAllocator.createTmpFileForWrite(name,
> 2 * blockSize,
> conf);
> return result;
> }
> {code}
> h3. LocalDirAllocator
> src/java/org/apache/hadoop/fs/LocalDirAllocator.java#AllocatorPerContext.java
> {code:java}
> /** Creates a file on the local FS. Pass size as -1 if not known apriori. We
> * round-robin over the set of disks (via the configured dirs) and return
> * a file on the first path which has enough space. The file is guaranteed
> * to go away when the JVM exits.
> */
> public File createTmpFileForWrite(String pathStr, long size,
> Configuration conf) throws IOException {
> // find an appropriate directory
> Path path = getLocalPathForWrite(pathStr, size, conf);
> File dir = new File(path.getParent().toUri().getPath());
> String prefix = path.getName();
> // create a temp file on this directory
> File result = File.createTempFile(prefix, null, dir);
> result.deleteOnExit();
> return result;
> }
> {code}
> First it seems to me it's a bit of a mess here I don't know if it's DFSClient.java#deleteBackupFile() or LocalDirAllocator#createTmpFileForWrite() {deleteOnExit(); ) who is call ... or both. Why not keep it dry and delete it only once.
> But the most important is the "deleteOnExit();" since it mean if it is never restarted it will never delete files :(
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2561)
/tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
Posted by "Benjamin Francisoud (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Francisoud updated HADOOP-2561:
----------------------------------------
Attachment: patch-DFSClient-HADOOP-2561.diff
The problem occur when an exception is thrown in the endBlock() method.
We never reach the deleteBackupFile() at the end of the method.
I added a try{} finally{} block to make sure we reach that code.
you can remove the LOG.info if you want.
> /tmp/hadoop-${user}/dfs/tmp/tmp/client-${long}.tmp is not cleanup correctly
> ---------------------------------------------------------------------------
>
> Key: HADOOP-2561
> URL: https://issues.apache.org/jira/browse/HADOOP-2561
> Project: Hadoop
> Issue Type: Bug
> Components: dfs, fs
> Affects Versions: 0.14.0
> Reporter: Benjamin Francisoud
> Attachments: patch-DFSClient-HADOOP-2561.diff
>
>
> Diretory "/tmp/hadoop-${user}/dfs/tmp/tmp" is being filled with those kinfd of files: client-226966559287638337420857.tmp
> I tried to look at the code and found:
> h3. DFSClient.java
> src/java/org/apache/hadoop/dfs/DFSClient.java
> {code:java}
> private void closeBackupStream() throws IOException {...}
> /* Similar to closeBackupStream(). Theoritically deleting a file
> * twice could result in deleting a file that we should not.
> */
> private void deleteBackupFile() {...}
> private File newBackupFile() throws IOException {
> String name = "tmp" + File.separator +
> "client-" + Math.abs(r.nextLong());
> File result = dirAllocator.createTmpFileForWrite(name,
> 2 * blockSize,
> conf);
> return result;
> }
> {code}
> h3. LocalDirAllocator
> src/java/org/apache/hadoop/fs/LocalDirAllocator.java#AllocatorPerContext.java
> {code:java}
> /** Creates a file on the local FS. Pass size as -1 if not known apriori. We
> * round-robin over the set of disks (via the configured dirs) and return
> * a file on the first path which has enough space. The file is guaranteed
> * to go away when the JVM exits.
> */
> public File createTmpFileForWrite(String pathStr, long size,
> Configuration conf) throws IOException {
> // find an appropriate directory
> Path path = getLocalPathForWrite(pathStr, size, conf);
> File dir = new File(path.getParent().toUri().getPath());
> String prefix = path.getName();
> // create a temp file on this directory
> File result = File.createTempFile(prefix, null, dir);
> result.deleteOnExit();
> return result;
> }
> {code}
> First it seems to me it's a bit of a mess here I don't know if it's DFSClient.java#deleteBackupFile() or LocalDirAllocator#createTmpFileForWrite() {deleteOnExit(); ) who is call ... or both. Why not keep it dry and delete it only once.
> But the most important is the "deleteOnExit();" since it mean if it is never restarted it will never delete files :(
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.