You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org> on 2018/09/27 01:33:47 UTC

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats 

Vuk Ercegovac has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11534


Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................

IMPALA-7622: adds profile metrics when fetching incremental stats

When computing incremental statistics by fetching the stats directly
from catalogd, a potentially expensive RPC is made from the impalad
coordinator to catalogd. This change adds metrics to the frontend
section of the profile to track how long the request takes, the size
of the compressed bytes received, and the number of partitions received.

The profile for a 'compute incremental ...' command on a table with
no statistics looks like this:

Frontend:
     - StatsFetch.Bytes: 0
     - StatsFetch.NumPartitionsReceived: 24
     - StatsFetch.NumPartitionsWithStats: 0
     - StatsFetch.Time: 26ms

And the profile looks as follows when the table has stats, so the stats
are fetched:

Frontend:
     - StatsFetch.Bytes: 24622
     - StatsFetch.NumPartitionsReceived: 23
     - StatsFetch.NumPartitionsWithStats: 23
     - StatsFetch.Time: 14ms

Testing:
- manual inspection
- e2e test to check the profile

Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
---
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_pull_stats.py
3 files changed, 72 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/11534/1
-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Sep 2018 11:22:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/835/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Sep 2018 02:20:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 1:

(8 comments)

http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java:

http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@651
PS1, Line 651:     Stopwatch sw = new Stopwatch().start();
> Should we break it down further into RPC fetch time and decompression time?
I don't think that level of detail is needed right now.


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@675
PS1, Line 675: numPartitionsReceived
> 'received' seems like an internal terminology due to out of sync coordinato
done. changed the metric name to TotalPartitions as well.


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@683
PS1, Line 683: partStatsFromCompressedBytes
> Also include decompressed bytes size from this method?
clarified to compressedByteSize.
for decompressed, that size is inconvenient to get at the moment (its hidden in that util) and don't we really want the size of the deserialized object, not just the bytes?


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@696
PS1, Line 696:       recordFetchMetrics(numBytes, numPartitionsReceived, numPartitionsWithStats, sw);
> nit: should we skip if an exception was thrown?
my thought here was that time/effort was spent in that rpc so lets report what we have.


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@700
PS1, Line 700:   private static void recordFetchMetrics(int numBytes, int numPartitionsReceived,
> method doc.
Done


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@704
PS1, Line 704: NONE
> BYTES for pretty printing?
Done


http://gerrit.cloudera.org:8080/#/c/11534/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/11534/1/tests/common/custom_cluster_test_suite.py@190
PS1, Line 190: i
> ouch
Done


http://gerrit.cloudera.org:8080/#/c/11534/1/tests/custom_cluster/test_pull_stats.py
File tests/custom_cluster/test_pull_stats.py:

http://gerrit.cloudera.org:8080/#/c/11534/1/tests/custom_cluster/test_pull_stats.py@64
PS1, Line 64:       # and should have been fetched.
> Add another partition and run incremental stats again and assert sure total
this functionality is tested in test_pull_stats. but added a test for this case for the sake of capturing it in the profile.



-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Sep 2018 19:06:21 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11534

to look at the new patch set (#3).

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................

IMPALA-7622: adds profile metrics when fetching incremental stats

When computing incremental statistics by fetching the stats directly
from catalogd, a potentially expensive RPC is made from the impalad
coordinator to catalogd. This change adds metrics to the frontend
section of the profile to track how long the request takes, the size
of the compressed bytes received, and the number of partitions received.

The profile for a 'compute incremental ...' command on a table with
no statistics looks like this:

Frontend:
     - StatsFetch.CompressedBytes: 0
     - StatsFetch.TotalPartitions: 24
     - StatsFetch.NumPartitionsWithStats: 0
     - StatsFetch.Time: 26ms

And the profile looks as follows when the table has stats, so the stats
are fetched:

Frontend:
     - StatsFetch.CompressedBytes: 24622
     - StatsFetch.TotalPartitions: 23
     - StatsFetch.NumPartitionsWithStats: 23
     - StatsFetch.Time: 14ms

Testing:
- manual inspection
- e2e test to check the profile

Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
---
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_pull_stats.py
3 files changed, 93 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/11534/3
-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................

IMPALA-7622: adds profile metrics when fetching incremental stats

When computing incremental statistics by fetching the stats directly
from catalogd, a potentially expensive RPC is made from the impalad
coordinator to catalogd. This change adds metrics to the frontend
section of the profile to track how long the request takes, the size
of the compressed bytes received, and the number of partitions received.

The profile for a 'compute incremental ...' command on a table with
no statistics looks like this:

Frontend:
     - StatsFetch.CompressedBytes: 0
     - StatsFetch.TotalPartitions: 24
     - StatsFetch.NumPartitionsWithStats: 0
     - StatsFetch.Time: 26ms

And the profile looks as follows when the table has stats, so the stats
are fetched:

Frontend:
     - StatsFetch.CompressedBytes: 24622
     - StatsFetch.TotalPartitions: 23
     - StatsFetch.NumPartitionsWithStats: 23
     - StatsFetch.Time: 14ms

Testing:
- manual inspection
- e2e test to check the profile

Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Reviewed-on: http://gerrit.cloudera.org:8080/11534
Reviewed-by: Vuk Ercegovac <ve...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_pull_stats.py
3 files changed, 93 insertions(+), 3 deletions(-)

Approvals:
  Vuk Ercegovac: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 4
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/11534

to look at the new patch set (#2).

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................

IMPALA-7622: adds profile metrics when fetching incremental stats

When computing incremental statistics by fetching the stats directly
from catalogd, a potentially expensive RPC is made from the impalad
coordinator to catalogd. This change adds metrics to the frontend
section of the profile to track how long the request takes, the size
of the compressed bytes received, and the number of partitions received.

The profile for a 'compute incremental ...' command on a table with
no statistics looks like this:

Frontend:
     - StatsFetch.CompressedBytes: 0
     - StatsFetch.TotalPartitions: 24
     - StatsFetch.NumPartitionsWithStats: 0
     - StatsFetch.Time: 26ms

And the profile looks as follows when the table has stats, so the stats
are fetched:

Frontend:
     - StatsFetch.CompressedBytes: 24622
     - StatsFetch.TotalPartitions: 23
     - StatsFetch.NumPartitionsWithStats: 23
     - StatsFetch.Time: 14ms

Testing:
- manual inspection
- e2e test to check the profile

Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
---
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_pull_stats.py
3 files changed, 88 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/11534/2
-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/862/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Sep 2018 07:46:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 2: Code-Review+2

(2 comments)

http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java:

http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@683
PS1, Line 683: tats =
> clarified to compressedByteSize.
agreed.


http://gerrit.cloudera.org:8080/#/c/11534/2/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java:

http://gerrit.cloudera.org:8080/#/c/11534/2/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@122
PS2, Line 122: private static final String STATS_FETCH_TIME = STATS_FETCH_PREFIX + ".Time";
             :   private static final String STATS_FETCH_COMPRESSED_BYTES =
             :       STATS_FETCH_PREFIX + ".CompressedBytes";
             :   private static final String STATS_FETCH_TOTAL_PARTITIONS =
             :       STATS_FETCH_PREFIX + ".TotalPartitions";
             :   private static final String STATS_FETCH_NUM_PARTITIONS_WITH_STATS =
             :       STATS_FETCH_PREFIX + ".NumPartitionsWithStats";
Sorry missed this in the first round. Would be nice to add one-liners that document what each of these metrics intends to collect.



-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Sep 2018 04:45:30 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 1:

(8 comments)

http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java:

http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@651
PS1, Line 651:     Stopwatch sw = new Stopwatch().start();
Should we break it down further into RPC fetch time and decompression time?


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@675
PS1, Line 675: numPartitionsReceived
'received' seems like an internal terminology due to out of sync coordinator and catalog. May be just say "totalPartitions" or something?

totalParts = partitions.size() (instead of incrementing inside the loop)


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@683
PS1, Line 683: partStatsFromCompressedBytes
Also include decompressed bytes size from this method?

StatsFetch.compressedByteSize
StatsFetch.decompressedByteSize


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@696
PS1, Line 696:       recordFetchMetrics(numBytes, numPartitionsReceived, numPartitionsWithStats, sw);
nit: should we skip if an exception was thrown?


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@700
PS1, Line 700:   private static void recordFetchMetrics(int numBytes, int numPartitionsReceived,
method doc.


http://gerrit.cloudera.org:8080/#/c/11534/1/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@704
PS1, Line 704: NONE
BYTES for pretty printing?


http://gerrit.cloudera.org:8080/#/c/11534/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/11534/1/tests/common/custom_cluster_test_suite.py@190
PS1, Line 190: i
ouch


http://gerrit.cloudera.org:8080/#/c/11534/1/tests/custom_cluster/test_pull_stats.py
File tests/custom_cluster/test_pull_stats.py:

http://gerrit.cloudera.org:8080/#/c/11534/1/tests/custom_cluster/test_pull_stats.py@64
PS1, Line 64:       # and should have been fetched.
Add another partition and run incremental stats again and assert sure totalParts and NumPartitionsWithStats counts?



-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Sep 2018 17:07:11 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 3: Code-Review+2

carry +2


-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Sep 2018 07:38:03 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/11534/2/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java:

http://gerrit.cloudera.org:8080/#/c/11534/2/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@122
PS2, Line 122: private static final String STATS_FETCH_TIME = STATS_FETCH_PREFIX + ".Time";
             :   private static final String STATS_FETCH_COMPRESSED_BYTES =
             :       STATS_FETCH_PREFIX + ".CompressedBytes";
             :   private static final String STATS_FETCH_TOTAL_PARTITIONS =
             :       STATS_FETCH_PREFIX + ".TotalPartitions";
             :   private static final String STATS_FETCH_NUM_PARTITIONS_WITH_STATS =
             :       STATS_FETCH_PREFIX + ".NumPartitionsWithStats";
> Sorry missed this in the first round. Would be nice to add one-liners that 
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Sep 2018 07:25:27 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/849/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Sep 2018 19:39:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7622: adds profile metrics when fetching incremental stats

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11534 )

Change subject: IMPALA-7622: adds profile metrics when fetching incremental stats
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3242/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 3
Gerrit-Owner: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 28 Sep 2018 07:47:17 +0000
Gerrit-HasComments: No