You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org> on 2021/03/19 21:05:01 UTC

[Impala-ASF-CR] IMPALA-10598: Deflake test cache reload validation

Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17210

Change subject: IMPALA-10598: Deflake test_cache_reload_validation
......................................................................

IMPALA-10598: Deflake test_cache_reload_validation

This patch deflakes the test test_cache_reload_validation in
test_hdfs_caching.py e2e test. The util method which the test relies on to
get the count of list of cache directives by parsing the output of command
"hdfs cacheadmin -listDirectives -stats" does not consider that the output
may contain trailing new lines or headers. Hence the test fails because the
expected number of cache directives does not match the number of lines
of the output.

The fix parses the line "Found <int> entries" in the output when available
and returns the count from that line. If the line is not found, it fallbacks
to the earlier implementation of using the number of lines.

Testing:
1. The test was failing for me when run individually. After the patch, I looped
the test 10 times without any errors.

Change-Id: I2d491e90af461d5db3575a5840958d17ca90901c
---
M tests/query_test/test_hdfs_caching.py
1 file changed, 18 insertions(+), 2 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/17210/1
--
To view, visit http://gerrit.cloudera.org:8080/17210
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2d491e90af461d5db3575a5840958d17ca90901c
Gerrit-Change-Number: 17210
Gerrit-PatchSet: 1
Gerrit-Owner: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-10598: Deflake test cache reload validation

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.

Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17210 )

Change subject: IMPALA-10598: Deflake test_cache_reload_validation
......................................................................


Patch Set 1: Code-Review+2

(2 comments)

LGTM. Thanks for the fix!

http://gerrit.cloudera.org:8080/#/c/17210/1/tests/query_test/test_hdfs_caching.py
File tests/query_test/test_hdfs_caching.py:

http://gerrit.cloudera.org:8080/#/c/17210/1/tests/query_test/test_hdfs_caching.py@349
PS1, Line 349: line for line in stdout.split('\n') if
             :       line.strip(' \n') != ''
nit: I think this can be simplified as

 [line for line in stdout.split('\n') if line.strip()]


http://gerrit.cloudera.org:8080/#/c/17210/1/tests/query_test/test_hdfs_caching.py@363
PS1, Line 363:     else:
             :       count = len(stdout.split('\n'))
             :     return count
nit: return len(stdout.split('\n'))



-- 
To view, visit http://gerrit.cloudera.org:8080/17210
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2d491e90af461d5db3575a5840958d17ca90901c
Gerrit-Change-Number: 17210
Gerrit-PatchSet: 1
Gerrit-Owner: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 23 Mar 2021 01:06:17 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10598: Deflake test cache reload validation

Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.

Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17210 )

Change subject: IMPALA-10598: Deflake test_cache_reload_validation
......................................................................

IMPALA-10598: Deflake test_cache_reload_validation

Testing:
1. The test was failing for me when run individually. After the patch, I looped
the test 10 times without any errors.

Change-Id: I2d491e90af461d5db3575a5840958d17ca90901c
---
M tests/query_test/test_hdfs_caching.py
1 file changed, 16 insertions(+), 2 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/17210/2
--
To view, visit http://gerrit.cloudera.org:8080/17210
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2d491e90af461d5db3575a5840958d17ca90901c
Gerrit-Change-Number: 17210
Gerrit-PatchSet: 2
Gerrit-Owner: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>