You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2008/09/19 20:10:44 UTC
[jira] Created: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
FSEditLog logs modification time instead of access time.
--------------------------------------------------------
Key: HADOOP-4225
URL: https://issues.apache.org/jira/browse/HADOOP-4225
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.19.0
Reporter: Konstantin Shvachko
Priority: Blocker
Fix For: 0.19.0
((FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632865#action_12632865 ]
dhruba borthakur commented on HADOOP-4225:
------------------------------------------
I am handling "append" as precisely the same as "opening the file at creation time and writing to it". I am assuming that every modification to a file also counts as an "access" to the file, hence every modification (including appends) should update the access time as well.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
>
> ((FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638192#action_12638192 ]
dhruba borthakur commented on HADOOP-4225:
------------------------------------------
This patch needs a disk format change. We have to guarantee that leases persisted by earlier versions of code are read correctly. I do not see the patch attached to this JIRA do these things. Am I missing something?
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638756#action_12638756 ]
Hudson commented on HADOOP-4225:
--------------------------------
Integrated in Hadoop-trunk #630 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/630/])
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633140#action_12633140 ]
dhruba borthakur commented on HADOOP-4225:
------------------------------------------
Are you saying that this is not a bug but more of a coding-structure convention? I am not able to understand the benefit of your approach.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4225:
----------------------------------------
Attachment: LogATime.patch
I am not arguing about the semantics of the access time field, I am saying that when saving it into the edits log you should save exactly that field rather than another one.
So, if you want to set accessTime to the value of the modificationTime you assign that value to that field. But when you serialize accessTime in {{logOpenFile()}} to the disk you should serialize that field rather than something else. I cannot even understand which of them (the first or the second) corresponds to which time until I look at the loading code.
This is what I mean.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> ((FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632850#action_12632850 ]
Konstantin Shvachko commented on HADOOP-4225:
---------------------------------------------
What about append?
Is it ok to change aTime to mTime during append. Or do we want to keep the original aTime?
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
>
> ((FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4225:
----------------------------------------
Description:
{{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
Introduced in HADOOP-1869.
was:
((FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
Introduced in HADOOP-1869.
Assignee: Konstantin Shvachko
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko resolved HADOOP-4225.
-----------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
I lust committed this.
I put it under improvements. Not a bug according to Dhruba.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638168#action_12638168 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4225:
------------------------------------------------
I think the correct way to fix this is changing logOpenFile(...) to write only mtime but not atime. However, this will introduce a layout change.
+1 to the patch. Konstantin's change makes the code more clear. Otherwise, it is hard to understand why writing the same number twice.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632840#action_12632840 ]
dhruba borthakur commented on HADOOP-4225:
------------------------------------------
Until the file is closed, the access time of a file is same as the modification time. Thus, access times need not be stored in the OP_ADD transaction. Neither does it need to be saved in the leases (INodeUnderConstruction).
The file close transaction OP_CLOSE needs to store access times, and it does.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
>
> ((FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4225) FSEditLog logs modification time
instead of access time.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638364#action_12638364 ]
Konstantin Shvachko commented on HADOOP-4225:
---------------------------------------------
The patch does not change neither the layout of data on the disk nor the values written. According to your previous comments the values of access and modification time are the same when the open file record is written.
> FSEditLog logs modification time instead of access time.
> --------------------------------------------------------
>
> Key: HADOOP-4225
> URL: https://issues.apache.org/jira/browse/HADOOP-4225
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: LogATime.patch
>
>
> {{FSEditLog.logOpenFile()}} does not persist accessTime but modificationTime instead. Should be {{getAccessTime()}}.
> Introduced in HADOOP-1869.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.