You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2010/09/23 19:54:34 UTC
[jira] Created: (HBASE-3033) [replication]
ReplicationSink.replicateEntries doesn't need to be sync'ed
[replication] ReplicationSink.replicateEntries doesn't need to be sync'ed
-------------------------------------------------------------------------
Key: HBASE-3033
URL: https://issues.apache.org/jira/browse/HBASE-3033
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.90.0
Once upon a time it was necessary to synchronize the access to ReplicationSink.replicateEntries but it's not required anymore. At the same time do the cleanup in the put/delete methods to de-dup the call this.pool.putTable(table);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3033) [replication]
ReplicationSink.replicateEntries doesn't need to be sync'ed
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-3033:
--------------------------------------
Attachment: HBASE-3033.patch
> [replication] ReplicationSink.replicateEntries doesn't need to be sync'ed
> -------------------------------------------------------------------------
>
> Key: HBASE-3033
> URL: https://issues.apache.org/jira/browse/HBASE-3033
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.90.0
>
> Attachments: HBASE-3033.patch
>
>
> Once upon a time it was necessary to synchronize the access to ReplicationSink.replicateEntries but it's not required anymore. At the same time do the cleanup in the put/delete methods to de-dup the call this.pool.putTable(table);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-3033) [replication]
ReplicationSink.replicateEntries improvements
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-3033.
---------------------------------------
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed to trunk and 0.89
> [replication] ReplicationSink.replicateEntries improvements
> -----------------------------------------------------------
>
> Key: HBASE-3033
> URL: https://issues.apache.org/jira/browse/HBASE-3033
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.89.20100924, 0.90.0
>
> Attachments: HBASE-3033.patch
>
>
> Once upon a time it was necessary to synchronize the access to ReplicationSink.replicateEntries but it's not required anymore. At the same time do the cleanup in the put/delete methods to de-dup the call this.pool.putTable(table);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3033) [replication]
ReplicationSink.replicateEntries improvements
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-3033:
--------------------------------------
Summary: [replication] ReplicationSink.replicateEntries improvements (was: [replication] ReplicationSink.replicateEntries doesn't need to be sync'ed)
Fix Version/s: 0.89.20100924
Changing title to be more accurate. Basically this jira:
- Removes the synchronization;
- Batches all the edits into one List<Put> per table; and
- Cleans up the duplicate call to putTable.
I'm also targeting this to 0.89.
> [replication] ReplicationSink.replicateEntries improvements
> -----------------------------------------------------------
>
> Key: HBASE-3033
> URL: https://issues.apache.org/jira/browse/HBASE-3033
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.89.20100924, 0.90.0
>
> Attachments: HBASE-3033.patch
>
>
> Once upon a time it was necessary to synchronize the access to ReplicationSink.replicateEntries but it's not required anymore. At the same time do the cleanup in the put/delete methods to de-dup the call this.pool.putTable(table);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3033) [replication]
ReplicationSink.replicateEntries doesn't need to be sync'ed
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914267#action_12914267 ]
Jean-Daniel Cryans commented on HBASE-3033:
-------------------------------------------
At the same time, instead of batching consecutive rows, we can do even better by creating a List<Put> per table and do a single call on HTable, leveraging MultiPut even more. I just rolled that out in production here I got 10x insertion speed (made a big enough difference that a region server that was piling up HLogs to replicate, was at about 450 each of 128MB, is now able to return back to live replication under the same write load).
> [replication] ReplicationSink.replicateEntries doesn't need to be sync'ed
> -------------------------------------------------------------------------
>
> Key: HBASE-3033
> URL: https://issues.apache.org/jira/browse/HBASE-3033
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.90.0
>
>
> Once upon a time it was necessary to synchronize the access to ReplicationSink.replicateEntries but it's not required anymore. At the same time do the cleanup in the put/delete methods to de-dup the call this.pool.putTable(table);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3033) [replication]
ReplicationSink.replicateEntries doesn't need to be sync'ed
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914638#action_12914638 ]
stack commented on HBASE-3033:
------------------------------
+1
> [replication] ReplicationSink.replicateEntries doesn't need to be sync'ed
> -------------------------------------------------------------------------
>
> Key: HBASE-3033
> URL: https://issues.apache.org/jira/browse/HBASE-3033
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.90.0
>
> Attachments: HBASE-3033.patch
>
>
> Once upon a time it was necessary to synchronize the access to ReplicationSink.replicateEntries but it's not required anymore. At the same time do the cleanup in the put/delete methods to de-dup the call this.pool.putTable(table);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.