[jira] [Created] (AURORA-116) Improve efficiency of saving host attributes (or avoid saving host attributes)

["Bill Farner (JIRA)" <ji...@apache.org>]

Bill Farner created AURORA-116:
----------------------------------

             Summary: Improve efficiency of saving host attributes (or avoid saving host attributes)
                 Key: AURORA-116
                 URL: https://issues.apache.org/jira/browse/AURORA-116
             Project: Aurora
          Issue Type: Task
          Components: Scheduler
            Reporter: Bill Farner
            Priority: Critical


The scheduler performs multiple write operations for every resource offer, to save slave attributes:
{noformat}
  public void resourceOffers(SchedulerDriver driver, List<Offer> offers) {
    Preconditions.checkState(registered, "Must be registered before receiving offers.");

    for (final Offer offer : offers) {
      log(Level.FINE, "Received offer: %s", offer);
      resourceOffers.incrementAndGet();
      storage.write(new MutateWork.NoResult.Quiet() {
        @Override protected void execute(MutableStoreProvider storeProvider) {
          storeProvider.getAttributeStore().saveHostAttributes(Conversions.getAttributes(offer));
        }
      });
{noformat}

This can unnecessarily block the singly-threaded message dispatch in the scheduler driver.  An incremental improvement would be to aggregate all slave info and save it in one write operation.  Better yet would be to perform writes asynchronously (taking care to not break task scheduling, since attributes are expected to be present).  Even better yet, it would be great to determine if we can avoid storing host attributes.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)