You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "radford nguyen (Code Review)" <ge...@cloudera.org> on 2019/04/04 22:39:36 UTC

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir 

radford nguyen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12935


Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................

IMPALA-8363: fix e2e start with impala_log_dir

This commit fixes the `CustomClusterTestSuite` to wait for the
correct number of executors when `impala_log_dir` is specified
in the test decorator.  Previously, the default value of 3
was always used, regardless of `cluster_size`.

Testing:

- Ran tests/authorization/test_ranger.py with custom
`impala_log_dir` and `cluster_size` arguments and
observed pass.  Observed to fail before changes

- Ran all original e2e tests

Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
---
M tests/common/custom_cluster_test_suite.py
1 file changed, 2 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/12935/1
-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................

IMPALA-8363: Fix E2E start with impala_log_dir

This commit fixes the `CustomClusterTestSuite` to wait for the
correct number of executors when `impala_log_dir` is specified
in the test decorator.  Previously, the default value of 3
was always used, regardless of `cluster_size`.

Testing:

- Manual verification using tests/authorization/test_ranger.py
with custom `impala_log_dir` and `cluster_size` arguments.
Failed before changes, passed after changes

- Ran all original E2E tests

Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Reviewed-on: http://gerrit.cloudera.org:8080/12935
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/common/custom_cluster_test_suite.py
1 file changed, 8 insertions(+), 8 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 4
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/12935/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/12935/1//COMMIT_MSG@7
PS1, Line 7: fix e2e start with impala_log_dir
nit: update the title to "Fix E2E start Impala cluster when impala_log_dir parameter is set"?


http://gerrit.cloudera.org:8080/#/c/12935/1//COMMIT_MSG@16
PS1, Line 16: tests/authorization/test_ranger.py
I think this file has nothing to do with the bug. Maybe mention that you manually updated this file to test the bug and verify the bug fix.



-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:08:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

Just couple nits. LGTM.


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:09:03 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2656/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:03:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

That would work for me


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Tue, 09 Apr 2019 21:37:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "radford nguyen (Code Review)" <ge...@cloudera.org>.
radford nguyen has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

> Patch Set 1:
> 
> (1 comment)

I think the kwargs approach is better because then `start_impala_cluster` can still be "in charge" of defaulting the `impala_log_dir` option


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Tue, 09 Apr 2019 18:44:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4012/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 3
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 16:12:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py@147
PS1, Line 147:     if IMPALA_LOG_DIR in method.func_dict:
             :       self._start_impala_cluster(cluster_args,
             :           default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
             :           impala_log_dir=method.func_dict[IMPALA_LOG_DIR], cluster_size=cluster_size,
             :           num_coordinators=cluster_size, expected_num_executors=cluster_size)
             :     else:
             :       self._start_impala_cluster(cluster_args,
             :           default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
             :           cluster_size=cluster_size, num_coordinators=cluster_size,
             :           expected_num_executors=cluster_size)
> i think this code may end up getting inconsistent if add more parameter. Ma
Agree with Fredy. Could also do something clever with kwargs, e.g. 

 kwargs = {}
 if IMPALA_LOG_DIR in method.func_dict:
   kwargs["impala_log_dir"] = ...
 self._start_impala_cluster(cluster_args, default_query_options=..., 
    **kwargs)

I'm just bikeshedding at this point though, Fredy's suggestion works fine.



-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:51:07 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2731/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 2
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 05:08:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "radford nguyen (Code Review)" <ge...@cloudera.org>.
Hello Austin Nobis, Fredy Wijaya, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/12935

to look at the new patch set (#2).

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................

IMPALA-8363: Fix E2E start with impala_log_dir

This commit fixes the `CustomClusterTestSuite` to wait for the
correct number of executors when `impala_log_dir` is specified
in the test decorator.  Previously, the default value of 3
was always used, regardless of `cluster_size`.

Testing:

- Manual verification using tests/authorization/test_ranger.py
with custom `impala_log_dir` and `cluster_size` arguments.
Failed before changes, passed after changes

- Ran all original E2E tests

Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
---
M tests/common/custom_cluster_test_suite.py
1 file changed, 8 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/12935/2
-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 2
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 3
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 21:19:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "radford nguyen (Code Review)" <ge...@cloudera.org>.
radford nguyen has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py@147
PS1, Line 147:     kwargs = {
             :       "cluster_size": cluster_size,
             :       "num_coordinators": cluster_size,
             :       "expected_num_executors": cluster_size,
             :       "default_query_options": method.func_dict.get(DEFAULT_QUERY_OPTIONS)
             :     }
             :     if IMPALA_LOG_DIR in method.func_dict:
             :       kwargs["impala_log_dir"] = method.func_dict[IMPALA_LOG_DIR]
             :     self._start_impala_cluster(cluster_args, **kwargs)
             :     super(CustomClusterTestSuite, self).setup_
> Agree with Fredy. Could also do something clever with kwargs, e.g. 
I think the kwargs approach is better because then `start_impala_cluster` can still be "in charge" of defaulting the `impala_log_dir` option



-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 2
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 04:27:06 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 2: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 2
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 16:11:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 2:

Thanks for the fix.


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 2
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 16:12:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: Fix E2E start with impala log dir

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: Fix E2E start with impala_log_dir
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 3
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 11 Apr 2019 16:12:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8363: fix e2e start with impala log dir

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12935 )

Change subject: IMPALA-8363: fix e2e start with impala_log_dir
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/12935/1/tests/common/custom_cluster_test_suite.py@147
PS1, Line 147:     if IMPALA_LOG_DIR in method.func_dict:
             :       self._start_impala_cluster(cluster_args,
             :           default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
             :           impala_log_dir=method.func_dict[IMPALA_LOG_DIR], cluster_size=cluster_size,
             :           num_coordinators=cluster_size, expected_num_executors=cluster_size)
             :     else:
             :       self._start_impala_cluster(cluster_args,
             :           default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
             :           cluster_size=cluster_size, num_coordinators=cluster_size,
             :           expected_num_executors=cluster_size)
i think this code may end up getting inconsistent if add more parameter. Maybe we should keep a single call instead.

      DEFAULT_IMPALA_LOG_DIR = os.getenv('LOG_DIR', "/tmp/")
      impala_log_dir = method.func_dict[IMPALA_LOG_DIR] if IMPALA_LOG_DIR else DEFAULT_IMPALA_LOG_DIR
      self._start_impala_cluster(cluster_args,
          default_query_options=method.func_dict.get(DEFAULT_QUERY_OPTIONS),
          impala_log_dir=impala_log_dir, cluster_size=cluster_size,
          num_coordinators=cluster_size, expected_num_executors=cluster_size)



-- 
To view, visit http://gerrit.cloudera.org:8080/12935
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f46f40474b4b380abe88647a37e8e4d2231d745
Gerrit-Change-Number: 12935
Gerrit-PatchSet: 1
Gerrit-Owner: radford nguyen <ra...@gmail.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 23:33:48 +0000
Gerrit-HasComments: Yes