[jira] [Comment Edited] (ACCUMULO-2889) Batch metadata table updates for new walogs

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027009#comment-14027009 ] 

Josh Elser edited comment on ACCUMULO-2889 at 6/10/14 9:15 PM:
---------------------------------------------------------------

Our current proposal:

{noformat}
TabletServer client threads:
order(commitSessions) // this is to avoid deadlock across multiple client threads

batch.start
foreach tablet:
  tablet.logLock.lock
  if (tablet.mustRegisterNewLoggers)
    then
      defineTablet(tablet) // write WAL entry for tablet
      tablet.addLoggerToMetadataBatch(batch)
      // hold onto the lock
    else
      tablet.logLock.release

batch.flush
release(allCurrentlyHeldLocks);
{noformat}

(edit: added some formatting to make this more readable)


was (Author: parkjsung):
Our current proposal:

TabletServer client threads:
order(commitSessions) // this is to avoid deadlock across multiple client threads

batch.start
foreach tablet:
  tablet.logLock.lock
  if (tablet.mustRegisterNewLoggers)
    then
      defineTablet(tablet) // write WAL entry for tablet
      tablet.addLoggerToMetadataBatch(batch)
      // hold onto the lock
    else
      tablet.logLock.release

batch.flush
release(allCurrentlyHeldLocks);

> Batch metadata table updates for new walogs
> -------------------------------------------
>
>                 Key: ACCUMULO-2889
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2889
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.5.1
>            Reporter: Jonathan Park
>
> Currently, when we update the Metadata table with new loggers, we will update the metadata for each tablet serially. We could optimize this to instead use a batchwriter to send all metadata updates for all tablets in a batch.
> A few special cases include:
> - What if the !METADATA tablet was included in the batch?
> - What about the root tablet?
> Benefit:
> In one of our clusters, we're experiencing particularly slow HDFS operations leading to large oscillations in ingest performance. We haven't isolated the cause in HDFS but when we profile the tservers, we noticed that they were waiting for metadata table operations to complete. This would target the waiting.
> Potential downsides:
> Given the existing locking scheme, it looks like we may have to lock a tablet for slightly longer (we'll lock for the duration of the batch).



--
This message was sent by Atlassian JIRA
(v6.2#6252)