You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/03/12 19:57:07 UTC
Review Request 31999: Querying For Requests By Task Status Has Poor
Performance
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31999/
-----------------------------------------------------------
Review request for Ambari, Nate Cole and Tom Beerbower.
Bugs: AMBARI-10050
https://issues.apache.org/jira/browse/AMBARI-10050
Repository: ambari
Description
-------
Ambari tries to retrieve Requests for a given RequestStatus (IN_PROGRESS, COMPLETED, FAILED) by using the sum of the collected states of all child task statuses.
The query that was being used was a mess.
Essentially, COMPLETED is the only status that is hard. The other statuses of IN_PROGRESS or FAILED just need to have a single task that is in those states, while COMPLETED means that _all_ tasks must be COMPLETED.
I broke this out into 2 different queries and got rid of the crazy cartesian product on the query with the nested SELECT.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java bd4f2d8
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java 7c7e8cd
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f9d21ad
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 9d538f7
Diff: https://reviews.apache.org/r/31999/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37:59 min
[INFO] Finished at: 2015-03-12T14:43:47-04:00
[INFO] Final Memory: 29M/461M
[INFO] ------------------------------------------------------------------------
Thanks,
Jonathan Hurley
Re: Review Request 31999: Querying For Requests By Task Status Has
Poor Performance
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31999/#review76274
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On March 12, 2015, 6:57 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31999/
> -----------------------------------------------------------
>
> (Updated March 12, 2015, 6:57 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-10050
> https://issues.apache.org/jira/browse/AMBARI-10050
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Ambari tries to retrieve Requests for a given RequestStatus (IN_PROGRESS, COMPLETED, FAILED) by using the sum of the collected states of all child task statuses.
>
> The query that was being used was a mess.
>
> Essentially, COMPLETED is the only status that is hard. The other statuses of IN_PROGRESS or FAILED just need to have a single task that is in those states, while COMPLETED means that _all_ tasks must be COMPLETED.
>
> I broke this out into 2 different queries and got rid of the crazy cartesian product on the query with the nested SELECT.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java bd4f2d8
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java 7c7e8cd
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f9d21ad
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 9d538f7
>
> Diff: https://reviews.apache.org/r/31999/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 37:59 min
> [INFO] Finished at: 2015-03-12T14:43:47-04:00
> [INFO] Final Memory: 29M/461M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 31999: Querying For Requests By Task Status Has
Poor Performance
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31999/#review76280
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On March 12, 2015, 4:01 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31999/
> -----------------------------------------------------------
>
> (Updated March 12, 2015, 4:01 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-10050
> https://issues.apache.org/jira/browse/AMBARI-10050
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Ambari tries to retrieve Requests for a given RequestStatus (IN_PROGRESS, COMPLETED, FAILED) by using the sum of the collected states of all child task statuses.
>
> The query that was being used was a mess.
>
> Essentially, COMPLETED is the only status that is hard. The other statuses of IN_PROGRESS or FAILED just need to have a single task that is in those states, while COMPLETED means that _all_ tasks must be COMPLETED.
>
> I broke this out into 2 different queries and got rid of the crazy cartesian product on the query with the nested SELECT.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java bd4f2d8
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java 7c7e8cd
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f9d21ad
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 9d538f7
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 1c16ce3
>
> Diff: https://reviews.apache.org/r/31999/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 37:59 min
> [INFO] Finished at: 2015-03-12T14:43:47-04:00
> [INFO] Final Memory: 29M/461M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 31999: Querying For Requests By Task Status Has
Poor Performance
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31999/
-----------------------------------------------------------
(Updated March 12, 2015, 4:01 p.m.)
Review request for Ambari, Nate Cole and Tom Beerbower.
Changes
-------
Saw that calls to getHosts(...) was actually starting and committing transactions, even when the hosts were stored in memory. Only the actual method that gets the hosts should be transactional since we short-circuit after initially retrieving them.
Bugs: AMBARI-10050
https://issues.apache.org/jira/browse/AMBARI-10050
Repository: ambari
Description
-------
Ambari tries to retrieve Requests for a given RequestStatus (IN_PROGRESS, COMPLETED, FAILED) by using the sum of the collected states of all child task statuses.
The query that was being used was a mess.
Essentially, COMPLETED is the only status that is hard. The other statuses of IN_PROGRESS or FAILED just need to have a single task that is in those states, while COMPLETED means that _all_ tasks must be COMPLETED.
I broke this out into 2 different queries and got rid of the crazy cartesian product on the query with the nested SELECT.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java bd4f2d8
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java 7c7e8cd
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f9d21ad
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 9d538f7
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 1c16ce3
Diff: https://reviews.apache.org/r/31999/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37:59 min
[INFO] Finished at: 2015-03-12T14:43:47-04:00
[INFO] Final Memory: 29M/461M
[INFO] ------------------------------------------------------------------------
Thanks,
Jonathan Hurley
Re: Review Request 31999: Querying For Requests By Task Status Has
Poor Performance
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31999/#review76273
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java
<https://reviews.apache.org/r/31999/#comment123788>
Nice, makes it clean.
- Alejandro Fernandez
On March 12, 2015, 6:57 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31999/
> -----------------------------------------------------------
>
> (Updated March 12, 2015, 6:57 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-10050
> https://issues.apache.org/jira/browse/AMBARI-10050
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Ambari tries to retrieve Requests for a given RequestStatus (IN_PROGRESS, COMPLETED, FAILED) by using the sum of the collected states of all child task statuses.
>
> The query that was being used was a mess.
>
> Essentially, COMPLETED is the only status that is hard. The other statuses of IN_PROGRESS or FAILED just need to have a single task that is in those states, while COMPLETED means that _all_ tasks must be COMPLETED.
>
> I broke this out into 2 different queries and got rid of the crazy cartesian product on the query with the nested SELECT.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java bd4f2d8
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java 7c7e8cd
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f9d21ad
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RequestDAO.java 9d538f7
>
> Diff: https://reviews.apache.org/r/31999/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 37:59 min
> [INFO] Finished at: 2015-03-12T14:43:47-04:00
> [INFO] Final Memory: 29M/461M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>