You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2015/10/29 15:48:54 UTC
Review Request 39758: Add normalizer properties for AMS Regions
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/
-----------------------------------------------------------
Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
Bugs: AMBARI-13632
https://issues.apache.org/jira/browse/AMBARI-13632
Repository: ambari
Description
-------
Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
Use the load simulator for these tests
Objective is to find out how often to run normalizer as a function of (sinks/hosts)
Make stack advisor changes as needed
Diffs
-----
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
Diff: https://reviews.apache.org/r/39758/diff/
Testing
-------
Unit tests passed
Thanks,
Dmytro Sen
Re: Review Request 39758: Add normalizer properties for AMS Regions
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/#review104451
-----------------------------------------------------------
Ship it!
Ship It!
- Sid Wagle
On Oct. 29, 2015, 2:59 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39758/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
>
>
> Bugs: AMBARI-13632
> https://issues.apache.org/jira/browse/AMBARI-13632
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
> Use the load simulator for these tests
> Objective is to find out how often to run normalizer as a function of (sinks/hosts)
> Make stack advisor changes as needed
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
>
> Diff: https://reviews.apache.org/r/39758/diff/
>
>
> Testing
> -------
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 39758: Add normalizer properties for AMS Regions
Posted by Sid Wagle <sw...@hortonworks.com>.
> On Oct. 29, 2015, 4:01 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector, line 34
> > <https://reviews.apache.org/r/39758/diff/2/?file=1112158#file1112158line34>
> >
> > We should do a little more analysis of how expensive this is, my initial though was only choose 3 tables : METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE.
> >
> > Better if we can make this configurable, we can add an export t0 ams-env.
>
> Dmytro Sen wrote:
> I think there is no need to make it configurable. Other tables are just skipped by the normalizer, they should have at least 3 regions to be considered for normalization.
> They are updated very rarely, so we can expect split/merge their regions once per month or even less often.
I see, ok lets make sure document that on the wiki. I will drop this issue.
> On Oct. 29, 2015, 4:01 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector, line 92
> > <https://reviews.apache.org/r/39758/diff/2/?file=1112158#file1112158line92>
> >
> > This should have some timeout value.
>
> Dmytro Sen wrote:
> alter_async is executed asyncronously, in backgound, so start command execution time isn't increased.
> We don't wait, that's a reason why we can't log if the altering table succeeded or not. If use syncronous alter command, collector start time can take up to 2 minutes.
>
> That's a workaround until normalization enabling supported by the Phoenix DDL
Very good, let me open a Jira for that to happen.
- Sid
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/#review104438
-----------------------------------------------------------
On Oct. 29, 2015, 2:59 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39758/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
>
>
> Bugs: AMBARI-13632
> https://issues.apache.org/jira/browse/AMBARI-13632
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
> Use the load simulator for these tests
> Objective is to find out how often to run normalizer as a function of (sinks/hosts)
> Make stack advisor changes as needed
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
>
> Diff: https://reviews.apache.org/r/39758/diff/
>
>
> Testing
> -------
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 39758: Add normalizer properties for AMS Regions
Posted by Dmytro Sen <ds...@hortonworks.com>.
> On Окт. 29, 2015, 4:01 п.п., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector, line 34
> > <https://reviews.apache.org/r/39758/diff/2/?file=1112158#file1112158line34>
> >
> > We should do a little more analysis of how expensive this is, my initial though was only choose 3 tables : METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE.
> >
> > Better if we can make this configurable, we can add an export t0 ams-env.
I think there is no need to make it configurable. Other tables are just skipped by the normalizer, they should have at least 3 regions to be considered for normalization.
They are updated very rarely, so we can expect split/merge their regions once per month or even less often.
> On Окт. 29, 2015, 4:01 п.п., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector, line 92
> > <https://reviews.apache.org/r/39758/diff/2/?file=1112158#file1112158line92>
> >
> > This should have some timeout value.
alter_async is executed asyncronously, in backgound, so start command execution time isn't increased.
We don't wait, that's a reason why we can't log if the altering table succeeded or not. If use syncronous alter command, collector start time can take up to 2 minutes.
That's a workaround until normalization enabling supported by the Phoenix DDL
- Dmytro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/#review104438
-----------------------------------------------------------
On Окт. 29, 2015, 2:59 п.п., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39758/
> -----------------------------------------------------------
>
> (Updated Окт. 29, 2015, 2:59 п.п.)
>
>
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
>
>
> Bugs: AMBARI-13632
> https://issues.apache.org/jira/browse/AMBARI-13632
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
> Use the load simulator for these tests
> Objective is to find out how often to run normalizer as a function of (sinks/hosts)
> Make stack advisor changes as needed
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
>
> Diff: https://reviews.apache.org/r/39758/diff/
>
>
> Testing
> -------
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 39758: Add normalizer properties for AMS Regions
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/#review104438
-----------------------------------------------------------
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector (line 34)
<https://reviews.apache.org/r/39758/#comment162696>
We should do a little more analysis of how expensive this is, my initial though was only choose 3 tables : METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE.
Better if we can make this configurable, we can add an export t0 ams-env.
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector (line 92)
<https://reviews.apache.org/r/39758/#comment162698>
This should have some timeout value.
- Sid Wagle
On Oct. 29, 2015, 2:59 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39758/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2015, 2:59 p.m.)
>
>
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
>
>
> Bugs: AMBARI-13632
> https://issues.apache.org/jira/browse/AMBARI-13632
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
> Use the load simulator for these tests
> Objective is to find out how often to run normalizer as a function of (sinks/hosts)
> Make stack advisor changes as needed
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
>
> Diff: https://reviews.apache.org/r/39758/diff/
>
>
> Testing
> -------
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 39758: Add normalizer properties for AMS Regions
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/
-----------------------------------------------------------
(Updated Окт. 29, 2015, 2:59 п.п.)
Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
Bugs: AMBARI-13632
https://issues.apache.org/jira/browse/AMBARI-13632
Repository: ambari
Description
-------
Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
Use the load simulator for these tests
Objective is to find out how often to run normalizer as a function of (sinks/hosts)
Make stack advisor changes as needed
Diffs (updated)
-----
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
Diff: https://reviews.apache.org/r/39758/diff/
Testing
-------
Unit tests passed
Thanks,
Dmytro Sen
Re: Review Request 39758: Add normalizer properties for AMS Regions
Posted by Myroslav Papirkovskyy <mp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39758/#review104427
-----------------------------------------------------------
Ship it!
Ship It!
- Myroslav Papirkovskyy
On Жов. 29, 2015, 4:48 після полудня, Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39758/
> -----------------------------------------------------------
>
> (Updated Жов. 29, 2015, 4:48 після полудня)
>
>
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
>
>
> Bugs: AMBARI-13632
> https://issues.apache.org/jira/browse/AMBARI-13632
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Test out the normalizer on AMS Regions for tables: METRIC_RECORD, METRIC_RECORD_MINUTE and METRIC_AGGREGATE
> Use the load simulator for these tests
> Objective is to find out how often to run normalizer as a function of (sinks/hosts)
> Make stack advisor changes as needed
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector dd7ee22
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 17f9b81
>
> Diff: https://reviews.apache.org/r/39758/diff/
>
>
> Testing
> -------
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>