You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "James Peach (JIRA)" <ji...@apache.org> on 2015/07/15 07:12:04 UTC
[jira] [Created] (MESOS-3051) performance issues with port ranges
comparison
James Peach created MESOS-3051:
----------------------------------
Summary: performance issues with port ranges comparison
Key: MESOS-3051
URL: https://issues.apache.org/jira/browse/MESOS-3051
Project: Mesos
Issue Type: Bug
Components: allocation
Affects Versions: 0.22.1
Reporter: James Peach
Testing in an environment with lots of frameworks (>200), where the frameworks permanently decline resources they don't need. The allocator ends up spending a lot of time figuring out whether offers are refused (the code path through {{HierarchicalAllocatorProcess::isFiltered()}}.
In profiling a synthetic benchmark, it turns out that comparing port ranges is very expensive, involving many temporary allocations. 61% of Resources::contains() run time is in operator -= (Resource). 35% of Resources::contains() run time is in Resources::_contains().
The heaviest call chain through {{Resources::_contains}} is:
{code}
Running Time Self (ms) Symbol Name
7237.0ms 35.5% 4.0 mesos::Resources::_contains(mesos::Resource const&) const
7200.0ms 35.3% 1.0 mesos::contains(mesos::Resource const&, mesos::Resource const&)
7133.0ms 35.0% 121.0 mesos::operator<=(mesos::Value_Ranges const&, mesos::Value_Ranges const&)
6319.0ms 31.0% 7.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Ranges const&)
6240.0ms 30.6% 161.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
1867.0ms 9.1% 25.0 mesos::Value_Ranges::add_range()
1694.0ms 8.3% 4.0 mesos::Value_Ranges::~Value_Ranges()
1495.0ms 7.3% 16.0 mesos::Value_Ranges::operator=(mesos::Value_Ranges const&)
445.0ms 2.1% 94.0 mesos::Value_Range::MergeFrom(mesos::Value_Range const&)
154.0ms 0.7% 24.0 mesos::Value_Ranges::range(int) const
103.0ms 0.5% 24.0 mesos::Value_Ranges::range_size() const
95.0ms 0.4% 2.0 mesos::Value_Range::Value_Range(mesos::Value_Range const&)
59.0ms 0.2% 4.0 mesos::Value_Ranges::Value_Ranges()
50.0ms 0.2% 50.0 mesos::Value_Range::begin() const
28.0ms 0.1% 28.0 mesos::Value_Range::end() const
26.0ms 0.1% 0.0 mesos::Value_Range::~Value_Range()
{code}
mesos::coalesce(Value_Ranges) gets done a lot and ends up being really expensive. The heaviest parts of the inverted call chain are:
{code}
Running Time Self (ms) Symbol Name
3209.0ms 15.7% 3209.0 mesos::Value_Range::~Value_Range()
3209.0ms 15.7% 0.0 google::protobuf::internal::GenericTypeHandler<mesos::Value_Range>::Delete(mesos::Value_Range*)
3209.0ms 15.7% 0.0 void google::protobuf::internal::RepeatedPtrFieldBase::Destroy<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>()
3209.0ms 15.7% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::~RepeatedPtrField()
3209.0ms 15.7% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::~RepeatedPtrField()
3209.0ms 15.7% 0.0 mesos::Value_Ranges::~Value_Ranges()
3209.0ms 15.7% 0.0 mesos::Value_Ranges::~Value_Ranges()
2441.0ms 11.9% 0.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
452.0ms 2.2% 0.0 mesos::remove(mesos::Value_Ranges*, mesos::Value_Range const&)
169.0ms 0.8% 0.0 mesos::operator<=(mesos::Value_Ranges const&, mesos::Value_Ranges const&)
82.0ms 0.4% 0.0 mesos::operator-=(mesos::Value_Ranges&, mesos::Value_Ranges const&)
65.0ms 0.3% 0.0 mesos::Value_Ranges::~Value_Ranges()
2541.0ms 12.4% 2541.0 google::protobuf::internal::GenericTypeHandler<mesos::Value_Range>::New()
2541.0ms 12.4% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>()
2305.0ms 11.3% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::Add()
2305.0ms 11.3% 0.0 mesos::Value_Ranges::add_range()
1962.0ms 9.6% 0.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
343.0ms 1.6% 0.0 mesos::ranges::add(mesos::Value_Ranges*, long long, long long)
236.0ms 1.1% 0.0 void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&)
1471.0ms 7.2% 1471.0 google::protobuf::internal::RepeatedPtrFieldBase::Reserve(int)
1333.0ms 6.5% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>()
1333.0ms 6.5% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::Add()
1333.0ms 6.5% 0.0 mesos::Value_Ranges::add_range()
1086.0ms 5.3% 0.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
247.0ms 1.2% 0.0 mesos::ranges::add(mesos::Value_Ranges*, long long, long long)
107.0ms 0.5% 0.0 void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&)
107.0ms 0.5% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::MergeFrom(google::protobuf::RepeatedPtrField<mesos::Value_Range> const&)
107.0ms 0.5% 0.0 mesos::Value_Ranges::MergeFrom(mesos::Value_Ranges const&)
105.0ms 0.5% 0.0 mesos::Value_Ranges::CopyFrom(mesos::Value_Ranges const&)
105.0ms 0.5% 0.0 mesos::Value_Ranges::operator=(mesos::Value_Ranges const&)
104.0ms 0.5% 0.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
1.0ms 0.0% 0.0 mesos::remove(mesos::Value_Ranges*, mesos::Value_Range const&)
2.0ms 0.0% 0.0 mesos::Resource::MergeFrom(mesos::Resource const&)
2.0ms 0.0% 0.0 google::protobuf::internal::GenericTypeHandler<mesos::Resource>::Merge(mesos::Resource const&, mesos::Resource*)
2.0ms 0.0% 0.0 void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom<google::protobuf::RepeatedPtrField<mesos::Resource>::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&)
29.0ms 0.1% 0.0 void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom<google::protobuf::RepeatedPtrField<mesos::Resource>::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&)
898.0ms 4.4% 898.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>()
517.0ms 2.5% 0.0 google::protobuf::RepeatedPtrField<mesos::Value_Range>::Add()
517.0ms 2.5% 0.0 mesos::Value_Ranges::add_range()
429.0ms 2.1% 0.0 mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
88.0ms 0.4% 0.0 mesos::ranges::add(mesos::Value_Ranges*, long long, long long)
379.0ms 1.8% 0.0 void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom<google::protobuf::RepeatedPtrField<mesos::Value_Range>::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)