You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (Created) (JIRA)" <ji...@apache.org> on 2012/04/04 21:56:19 UTC
[jira] [Created] (HDFS-3196) Implement JournalListener for writing
journal to local disk
Implement JournalListener for writing journal to local disk
-----------------------------------------------------------
Key: HDFS-3196
URL: https://issues.apache.org/jira/browse/HDFS-3196
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3196) Implement JournalListener for
writing journal to local disk
Posted by "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253755#comment-13253755 ]
Tsz Wo (Nicholas), SZE commented on HDFS-3196:
----------------------------------------------
Hari's patch in HDFS-3274 has editlog initialization. We should simply use it in JournalDiskWriter.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3196) Implement JournalListener for writing
journal to local disk
Posted by "Tsz Wo (Nicholas), SZE (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-3196:
-----------------------------------------
Attachment: h3196_20120416.patch
h3196_20120416.patch: adds a Journal class so that reader and writer can use it.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch, h3196_20120416.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3196) Implement JournalListener for writing
journal to local disk
Posted by "Tsz Wo (Nicholas), SZE (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-3196:
-----------------------------------------
Attachment: h3196_20120412.patch
h3196_20120412.patch:
- adds JournalDiskWriter;
- renames JournalListener.rollLogs(..) to startLogSegment(..).
Note that the editlog initialization will be done separately.
There are some code refactoring in the patch. I am think about doing it on trunk first.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3196) Implement JournalListener for
writing journal to local disk
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256023#comment-13256023 ]
Hari Mankude commented on HDFS-3196:
------------------------------------
Looks good
+1
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch, h3196_20120416.patch, h3196_20120417.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3196) Implement JournalListener for writing
journal to local disk
Posted by "Tsz Wo (Nicholas), SZE (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE resolved HDFS-3196.
------------------------------------------
Resolution: Fixed
Fix Version/s: Shared journals (HDFS-3092)
Hadoop Flags: Reviewed
Thanks Hari and Suresh for reviewing this.
I have committed this.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: Shared journals (HDFS-3092)
>
> Attachments: h3196_20120412.patch, h3196_20120416.patch, h3196_20120417.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3196) Implement JournalListener for
writing journal to local disk
Posted by "Suresh Srinivas (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253062#comment-13253062 ]
Suresh Srinivas commented on HDFS-3196:
---------------------------------------
Comments:
# JournalDiskWriter
#* Looks like it is incomplete. Please add TODO items at least.
#* When is editlog instantiated, you need directories where editlogs are written etc. Currently editlog is null.
#* Please add tests for JournalDiskWriter.
# I had renamed
# Unrelated to this patch, can you remove unneccessary IOException thrown frmo FSEditLog#namenodeStartedLogSegment.
# I prefer calling namenodeStartedLogSegment to startLogSegment and indicate in the comments that the remote namenode started a new log segment.
I like the idea of refactoring the code in trunk first and bringing it in.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3196) Implement JournalListener for
writing journal to local disk
Posted by "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253704#comment-13253704 ]
Tsz Wo (Nicholas), SZE commented on HDFS-3196:
----------------------------------------------
I created HDFS-3273 for refactoring. Then, I am going to work on editlog initialization here.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3196) Implement JournalListener for
writing journal to local disk
Posted by "Hari Mankude (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255816#comment-13255816 ]
Hari Mankude commented on HDFS-3196:
------------------------------------
Minor comments:
- JournalService should create the journal object.
- Please verify that in-memory structures are initialized correctly after the format() command
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch, h3196_20120416.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3196) Implement JournalListener for writing
journal to local disk
Posted by "Tsz Wo (Nicholas), SZE (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-3196:
-----------------------------------------
Attachment: h3196_20120417.patch
h3196_20120417.patch:
- removes Journal from the JournalService constructor;
- adds TestJournal for testing format.
> Implement JournalListener for writing journal to local disk
> -----------------------------------------------------------
>
> Key: HDFS-3196
> URL: https://issues.apache.org/jira/browse/HDFS-3196
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h3196_20120412.patch, h3196_20120416.patch, h3196_20120417.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira