[jira] [Commented] (AURORA-137) Save host attributes only when a task is being scheduled

["Mehrdad Nurolahzade (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AURORA-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15674805#comment-15674805 ] 

Mehrdad Nurolahzade commented on AURORA-137:
--------------------------------------------

I have noticed contention on storage write lock due to processing large collections of resource offers in {{MesosSchedulerImpl.resourceOffers()}}. 

Now that we have a general purpose {{BatchWorker}} in place. We could switch from the current implementation of {{executor}} to an implementation based on {{BatchWorker}} and tune {{maxBatchSize}} properly to increase concurrency.

> Save host attributes only when a task is being scheduled
> --------------------------------------------------------
>
>                 Key: AURORA-137
>                 URL: https://issues.apache.org/jira/browse/AURORA-137
>             Project: Aurora
>          Issue Type: Story
>          Components: Scheduler
>            Reporter: Bill Farner
>            Priority: Minor
>
> The scheduler currently aggressively saves host attributes when handling {{resourceOffers}}, however it seems tractable for this to only happen when a task is actually scheduled.  Context: the scheduler stores host attributes to satisfy scheduling constraints (like host/rack diversity).  Doing this would allow us to avoid waiting for the storage write lock, and handle {{resourceOffers}} in a more deterministic time frame.
> One caveat with this approach is that the Offer would need to be plumbed into {{SchedulingFilterImpl}} in a way so as to ensure that the attributes are available for the offer being inspected.  In other words, we need to avoid the chicken and egg of trying to read the attributes for a host when this is the first offer ever received for it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)