You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Chris Douglas (JIRA)" <ji...@apache.org> on 2008/10/07 02:25:44 UTC
[jira] Created: (HADOOP-4358) NPE from CreateEditsLog
NPE from CreateEditsLog
-----------------------
Key: HADOOP-4358
URL: https://issues.apache.org/jira/browse/HADOOP-4358
Project: Hadoop Core
Issue Type: Bug
Components: dfs, test
Affects Versions: 0.19.0
Reporter: Chris Douglas
Priority: Blocker
HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
{noformat}
java.lang.NullPointerException
at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640075#action_12640075 ]
dhruba borthakur commented on HADOOP-4358:
------------------------------------------
I agree with Konstantin that the using the precision as we do now is actually fine. But making the code simple sounds even better. I am +1 on Raghu's patch.
@Raghu: can you pl delete the comment "precise to the last hour" in INode.java? Thanks.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639541#action_12639541 ]
Konstantin Shvachko commented on HADOOP-4358:
---------------------------------------------
- True, {{accessTime}} field is integer rather than long, I forgot. Then there is a question what if somebody sets {{dfs.access.time.precision}} to 1 second or even 1 millisecond will that screw up the access time? Looks like that. We should change the time unit for {{dfs.access.time.precision}}. Now it is in milliseconds, we should measure it in hours.
- Returning to the issue. The unit test calls a copy constructor of INode, which calls setAssessTime(), which in turn calls static method {{FSNamesystem.getFSNamesystem().getAccessTimePrecision()}}. So the problem here is to get rid of the static method call. And you cannot do it in the unit test, because the INode constructor used there does not even specify the access time.
I agree with Nicholas that obtaining non-static members from FSNamesystem like that is not a good idea, it is more like a shortcut. We should change the INode code and add the precision parameter to all methods/constructors that set access time, and have a constant for the default precision.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640328#action_12640328 ]
Raghu Angadi commented on HADOOP-4358:
--------------------------------------
We caught one of the nodes in this state in last couple of minutes. 'sda' was very busy and the remaining disks were idle. Logs and os also reside on sda.. ssh login took more than a minute. This can explain why reading a block could take so long. Still not sure what causes the load on sda alone. we have datanode log for this node. Didn't have much time to take jstack and other info.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639999#action_12639999 ]
Raghu Angadi commented on HADOOP-4358:
--------------------------------------
> I don't think we can achieve this if we do not want to store the entire long for each file/directory object.
Even if you we want to keep it an 'int', truncating with precision does not seem useful in anyway. For ints, we could round it to seconds for e.g. My preferred solution is to just to keep it long and keep it simple.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Status: Open (was: Patch Available)
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Comment: was deleted
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639939#action_12639939 ]
Raghu Angadi commented on HADOOP-4358:
--------------------------------------
Fortunately, fsimage does not depend on precision. I hope we fix it as part of this jira.
INode should be totally unaware of precision.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sameer Paranjpye updated HADOOP-4358:
-------------------------------------
Fix Version/s: 0.19.0
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Release Note: No need to truncate access time in INode. Also fixes NPE in CreateEditsLog
Hadoop Flags: [Reviewed]
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642293#action_12642293 ]
Hudson commented on HADOOP-4358:
--------------------------------
Integrated in Hadoop-trunk #640 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/640/])
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639932#action_12639932 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4358:
------------------------------------------------
Konstantin> This will be inconsistent with getAccessTime(). Or do you propose to "unround" it back after getting access time.
Yes, I am proposing both getAccessTime() and setAccessTime(..) should use int.
When reading/writing access time to edits/images, precision could be passed as a parameter and atime is written as a long.
Dhruba> Is it possible to come up with a solution that allows me to have different precisions for two different clusters?
What do you mean by this? What operation you want to support? I think we cannot compare access times with different precisions in some cases. Supposed atime1="Oct 15, 11:00am" and atime2="Oct 15, 11:00am". However, atime1 and atime2 have precisions 1 hour and 1 minute, respectively. Then, we cannot tell whether atime1 is < or == or > atime2.
Dhruba> Ideally, the precision should be a filesystem constant and can change only when you format a HDFS filesystem.
I think we could safely allow decreasing precision. e.g. changing 1hour to 1day. For increasing precision, we should thrown an exception.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640633#action_12640633 ]
Hadoop QA commented on HADOOP-4358:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12392307/HADOOP-4358.patch
against trunk revision 705552.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3484/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3484/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3484/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3484/console
This message is automatically generated.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-4358:
-------------------------------------
Status: Patch Available (was: Open)
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639935#action_12639935 ]
Raghu Angadi commented on HADOOP-4358:
--------------------------------------
It seems like getAccessTime() is unnecessarily tied to "precision". Precision is just a configuration number to help reduce performance impact. "precision" of 1 hour should not mean all times should be at hour. Am I missing something?
I thought accessTime is just the time when it was 'updated' (currently this implies when it was written edits.log).
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-4358:
------------------------------------
Release Note: (was: No need to truncate access time in INode. Also fixes NPE in CreateEditsLog)
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639666#action_12639666 ]
Konstantin Shvachko commented on HADOOP-4358:
---------------------------------------------
> I think we should round off atime to int before passing to INode.
This will be inconsistent with getAccessTime(). Or do you propose to "unround" it back after getting access time.
The more I think about it the more inconsistent it appears. If we store a rounded aTime, then shutdown the cluster, and restart it with a different precision configuration parameter. Then the stored value does not make any sense. It looks like we cannot have the precision configured. It must be a constant. Is that right?
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Attachment: HADOOP-4358.patch
Does the attached patch fix this issue and 'precision' related confusion?
Changing 'int' to 'long' does not actually increase the memory.. because of alignment issues etc (especially on 64bit jvms).. even if we want to keep it int for some reason, we could just round it to a second rather than tying it to 'precision'
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640123#action_12640123 ]
Hadoop QA commented on HADOOP-4358:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12392199/HADOOP-4358.patch
against trunk revision 705155.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3473/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3473/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3473/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3473/console
This message is automatically generated.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-4358:
------------------------------------
Assignee: Raghu Angadi
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639622#action_12639622 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4358:
------------------------------------------------
> We should change the INode code and add the precision parameter to all methods/constructors that set access time, and have a constant for the default precision.
I think we should round off atime to int before passing to INode.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639254#action_12639254 ]
dhruba borthakur commented on HADOOP-4358:
------------------------------------------
In that case, you have to increase memory usage on the NameNode by another 4 bytes per inode ( just to pass this unit test).
Ca we change the unit test itself?
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639241#action_12639241 ]
Konstantin Shvachko commented on HADOOP-4358:
---------------------------------------------
{{CreateEditsLog}} generates edits file. It should be able to work without name-node present. Unfortunately, {{INode.setAccessTime()}} calls a static method on {{FSNamesystem}}, which had not been instantiated at the time.
I think the right thing to do here is to store aTime as-is, rather than up to an hour. {{getAccessTime()}} then should round aTime up to the latset hour and return it.
{code}
void setAccessTime(long aTime) {
this.accessTime = aTime;
}
long getAccessTime() {
return (this.accessTime / precision) * precision;
}
{code}
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639927#action_12639927 ]
dhruba borthakur commented on HADOOP-4358:
------------------------------------------
Is it possible to come up with a solution that allows me to have different precisions for two different clusters?
Ideally, the precision should be a filesystem constant and can change only when you format a HDFS filesystem.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637325#action_12637325 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4358:
------------------------------------------------
This is another case that we should not use static variables for non-constant values. See also
https://issues.apache.org/jira/browse/HADOOP-2413?focusedCommentId=12603150#action_12603150
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Status: Patch Available (was: Open)
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640348#action_12640348 ]
dhruba borthakur commented on HADOOP-4358:
------------------------------------------
+1 for the code changes.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639988#action_12639988 ]
Konstantin Shvachko commented on HADOOP-4358:
---------------------------------------------
Yes, the image stores the long value, that is we store the value returned by getAccessTime() and the precision is factored out at this point. It is just that the value is rounded up to the latest whole hour. So if we start a cluster with the new precision value then this rounded value will be divided by the new precision and everything is correct. So, there is no issue with configuring precision value as I suggested before, and Dhruba can safely run different clusters with different precisions. Sorry about the confusion.
# The question is how to get rid of the static method call in INode.
#- I think we should change setAccessTime to {{setAccessTime(lonk aTime, int precision)}} and propagate changes to the INode constructors.
#- Alternatively, we can create a {{static FSNmesystem.getPresision()}}, which would return the default precision if {{FSNmesystem.this == null}}. Don't really like it, seems like another shortcut to me.
# Second question is in what units the precision should be specified in configuration?
Aren't we introducing a Y2K-type bug even if the precision is specified it in hours?
# Raghu> _INode should be totally unaware of precision._
I don't think we can achieve this if we do not want to store the entire long for each file/directory object.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4358) NPE from CreateEditsLog
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4358:
---------------------------------
Attachment: HADOOP-4358.patch
Updated patch is attached. Removed a comment and unused import.
> can you pl delete the comment "precise to the last hour" in INode.java?
done.
> NPE from CreateEditsLog
> -----------------------
>
> Key: HADOOP-4358
> URL: https://issues.apache.org/jira/browse/HADOOP-4358
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Reporter: Chris Douglas
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4358.patch, HADOOP-4358.patch
>
>
> HADOOP-1869 added a call to setAccessTime(long) from the INode cstr, which relies on a non-null value from FSNamesystem::getFSNamesystem.
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hdfs.server.namenode.INode.setAccessTime(INode.java:301)
> at org.apache.hadoop.hdfs.server.namenode.INode.<init>(INode.java:99)
> at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.<init>(INodeDirectory.java:45)
> at org.apache.hadoop.hdfs.CreateEditsLog.addFiles(CreateEditsLog.java:68)
> at org.apache.hadoop.hdfs.CreateEditsLog.main(CreateEditsLog.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.