You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2020/03/27 14:15:23 UTC

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters 

Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15568


Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................

[test] Use `system_unsync` time for Kudu test clusters

Recently Kudu made enhancements to time source configuration and
adjusted the time source for it’s local clusters/tests to `system_unsync`.

This patch mirrors that behavior in Impala test clusters given there is no need to require NTP-synchronized clock for a test where all the
participating Kudu masters and tablet servers are run at the same node
using the same local wallclock.

See the Kudu commit here for details:
https://github.com/apache/kudu/commit/eb2b70d4b96be2fc2fdd6b3625acc284ac5774be

Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
---
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
2 files changed, 16 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15568/1
-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................

IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters

Recently Kudu made enhancements to time source configuration and
adjusted the time source for local clusters/tests to `system_unsync`.

This patch mirrors that behavior in Impala test clusters given there is no
need to require NTP-synchronized clock for a test where all the
participating Kudu masters and tablet servers are run at the same node
using the same local wallclock.

See the Kudu commit here for details:
https://github.com/apache/kudu/commit/eb2b70d4b96be2fc2fdd6b3625acc284ac5774be

While making this change, I removed all ntp related packages and special
handling as they should not be needed in a development environment
any more. I also added curl and gawk which were missing in my
Docker ubuntu environment and broke my testing.

Testing:
I tested with the steps below using Docker for Mac:

  docker rm impala-dev
  docker volume rm impala
  docker run --privileged --interactive --tty --name impala-dev -v impala:/home -p 25000:25000 -p 25010:25010 -p 25020:25020 ubuntu:16.04 /bin/bash

  apt-get update
  apt-get install sudo
  adduser --disabled-password --gecos '' impdev
  echo 'impdev ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
  su - impdev
  cd ~

  sudo apt-get --yes install git
  git clone https://git-wip-us.apache.org/repos/asf/impala.git ~/Impala
  cd ~/Impala
  export IMPALA_HOME=`pwd`
  git remote add fork https://github.com/granthenke/impala.git
  git fetch fork
  git checkout kudu-system-time

  $IMPALA_HOME/bin/bootstrap_development.sh

  source $IMPALA_HOME/bin/impala-config.sh
  (pushd fe && mvn -fae test -Dtest=AnalyzeDDLTest)
  (pushd fe && mvn -fae test -Dtest=AnalyzeKuduDDLTest)

  $IMPALA_HOME/bin/start-impala-cluster.py
  ./tests/run-tests.py query_test/test_kudu.py

Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Reviewed-on: http://gerrit.cloudera.org:8080/15568
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/bootstrap_system.sh
M docker/entrypoint.sh
M testdata/cluster/admin
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
5 files changed, 22 insertions(+), 57 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified
  Alexey Serbin: Looks good to me, but someone else must approve

-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5648/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 30 Mar 2020 22:10:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5619/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 14:55:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 19:30:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Alexey Serbin, Attila Bukor, Tim Armstrong, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15568

to look at the new patch set (#4).

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................

[test] Use `system_unsync` time for Kudu test clusters

Recently Kudu made enhancements to time source configuration and
adjusted the time source for local clusters/tests to `system_unsync`.

This patch mirrors that behavior in Impala test clusters given there is no
need to require NTP-synchronized clock for a test where all the
participating Kudu masters and tablet servers are run at the same node
using the same local wallclock.

See the Kudu commit here for details:
https://github.com/apache/kudu/commit/eb2b70d4b96be2fc2fdd6b3625acc284ac5774be

While making this change, I removed all ntp related packages and special
handling as they should not be needed in a development environment
any more. I also added curl and gawk which were missing in my
Docker ubuntu environment and broke my testing.

Testing:
I tested with the steps below using Docker for Mac:

  docker rm impala-dev
  docker volume rm impala
  docker run --privileged --interactive --tty --name impala-dev -v impala:/home -p 25000:25000 -p 25010:25010 -p 25020:25020 ubuntu:16.04 /bin/bash

  apt-get update
  apt-get install sudo
  adduser --disabled-password --gecos '' impdev
  echo 'impdev ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
  su - impdev
  cd ~

  sudo apt-get --yes install git
  git clone https://git-wip-us.apache.org/repos/asf/impala.git ~/Impala
  cd ~/Impala
  export IMPALA_HOME=`pwd`
  git remote add fork https://github.com/granthenke/impala.git
  git fetch fork
  git checkout kudu-system-time

  $IMPALA_HOME/bin/bootstrap_development.sh

  source $IMPALA_HOME/bin/impala-config.sh
  (pushd fe && mvn -fae test -Dtest=AnalyzeDDLTest)
  (pushd fe && mvn -fae test -Dtest=AnalyzeKuduDDLTest)

  $IMPALA_HOME/bin/start-impala-cluster.py
  ./tests/run-tests.py query_test/test_kudu.py

Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
---
M bin/bootstrap_system.sh
M docker/entrypoint.sh
M testdata/cluster/admin
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
5 files changed, 22 insertions(+), 57 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15568/4
-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 6: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 15:53:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 6:

I agree that the failure is unrelated, restarted the build and created a jira for it: IMPALA-9584


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 15:03:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 14:43:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15568/3/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/15568/3/bin/bootstrap_system.sh@227
PS3, Line 227: redhat sudo yum install -y curl gawk gcc gcc-c++ git krb5-devel krb5-server krb5-workstation \
line too long (94 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 17:32:47 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5584/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 15:02:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2:

(1 comment)

> I can make those other clean up changes. Mind if I do it in a
 > follow on patch?

Sure. Doesn't seem like a lot of work so putting it in this patch would be nice, but up to you.

http://gerrit.cloudera.org:8080/#/c/15568/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/15568/2//COMMIT_MSG@7
PS2, Line 7: [test]
Please file an IMPALA jira for this. If you split it up into two patches, you can use the same one, just mark the second review as (part 2)



-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 19:36:25 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 15:02:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 20:33:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 30 Mar 2020 21:31:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Alexey Serbin, Attila Bukor, Tim Armstrong, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15568

to look at the new patch set (#3).

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................

[test] Use `system_unsync` time for Kudu test clusters

Recently Kudu made enhancements to time source configuration and
adjusted the time source for local clusters/tests to `system_unsync`.

This patch mirrors that behavior in Impala test clusters given there is no
need to require NTP-synchronized clock for a test where all the
participating Kudu masters and tablet servers are run at the same node
using the same local wallclock.

See the Kudu commit here for details:
https://github.com/apache/kudu/commit/eb2b70d4b96be2fc2fdd6b3625acc284ac5774be

While making this change, I removed all ntp related packages and special
handling as they should not be needed in a development environment
any more. I also added curl and gawk which were missing in my
Docker ubuntu environment and broke my testing.

Testing:
I tested with the steps below using Docker for Mac:

  docker rm impala-dev
  docker volume rm impala
  docker run --privileged --interactive --tty --name impala-dev -v impala:/home -p 25000:25000 -p 25010:25010 -p 25020:25020 ubuntu:16.04 /bin/bash

  apt-get update
  apt-get install sudo
  adduser --disabled-password --gecos '' impdev
  echo 'impdev ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
  su - impdev
  cd ~

  sudo apt-get --yes install git
  git clone https://git-wip-us.apache.org/repos/asf/impala.git ~/Impala
  cd ~/Impala
  export IMPALA_HOME=`pwd`
  git remote add fork https://github.com/granthenke/impala.git
  git fetch fork
  git checkout kudu-system-time

  $IMPALA_HOME/bin/bootstrap_development.sh

  source $IMPALA_HOME/bin/impala-config.sh
  (pushd fe && mvn -fae test -Dtest=AnalyzeDDLTest)
  (pushd fe && mvn -fae test -Dtest=AnalyzeKuduDDLTest)

  $IMPALA_HOME/bin/start-impala-cluster.py
  ./tests/run-tests.py query_test/test_kudu.py

Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
---
M bin/bootstrap_system.sh
M docker/entrypoint.sh
M testdata/cluster/admin
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
5 files changed, 21 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15568/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5574/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 30 Mar 2020 21:32:01 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5555/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 14:56:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2:

I can make those other clean up changes. Mind if I do it in a follow on patch?


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 18:23:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2:

That is great news if we can remove the ntp requirement for our dev cluster.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 23:35:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2:

yes, we should be able to remove all ntp related steps.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 18:22:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5636/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 18:13:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 5: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 30 Mar 2020 22:23:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15568/3/bin/bootstrap_system.sh
File bin/bootstrap_system.sh:

http://gerrit.cloudera.org:8080/#/c/15568/3/bin/bootstrap_system.sh@227
PS3, Line 227: redhat sudo yum install -y curl gawk gcc gcc-c++ git krb5-devel krb5-server krb5-workstation \
> line too long (94 > 90)
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 17:36:57 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 22:45:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Alexey Serbin, Attila Bukor, Tim Armstrong, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15568

to look at the new patch set (#2).

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................

[test] Use `system_unsync` time for Kudu test clusters

Recently Kudu made enhancements to time source configuration and
adjusted the time source for it’s local clusters/tests to `system_unsync`.

This patch mirrors that behavior in Impala test clusters given there is no
need to require NTP-synchronized clock for a test where all the
participating Kudu masters and tablet servers are run at the same node
using the same local wallclock.

See the Kudu commit here for details:
https://github.com/apache/kudu/commit/eb2b70d4b96be2fc2fdd6b3625acc284ac5774be

Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
---
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
2 files changed, 16 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15568/2
-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 19:38:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 5:

It looks like some parquet analytic function tests failed in the verify job. I don't think that is related.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 13:33:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5637/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 18:17:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5564/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 18:14:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Alexey Serbin, Attila Bukor, Tim Armstrong, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15568

to look at the new patch set (#5).

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................

IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters

Recently Kudu made enhancements to time source configuration and
adjusted the time source for local clusters/tests to `system_unsync`.

This patch mirrors that behavior in Impala test clusters given there is no
need to require NTP-synchronized clock for a test where all the
participating Kudu masters and tablet servers are run at the same node
using the same local wallclock.

See the Kudu commit here for details:
https://github.com/apache/kudu/commit/eb2b70d4b96be2fc2fdd6b3625acc284ac5774be

While making this change, I removed all ntp related packages and special
handling as they should not be needed in a development environment
any more. I also added curl and gawk which were missing in my
Docker ubuntu environment and broke my testing.

Testing:
I tested with the steps below using Docker for Mac:

  docker rm impala-dev
  docker volume rm impala
  docker run --privileged --interactive --tty --name impala-dev -v impala:/home -p 25000:25000 -p 25010:25010 -p 25020:25020 ubuntu:16.04 /bin/bash

  apt-get update
  apt-get install sudo
  adduser --disabled-password --gecos '' impdev
  echo 'impdev ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
  su - impdev
  cd ~

  sudo apt-get --yes install git
  git clone https://git-wip-us.apache.org/repos/asf/impala.git ~/Impala
  cd ~/Impala
  export IMPALA_HOME=`pwd`
  git remote add fork https://github.com/granthenke/impala.git
  git fetch fork
  git checkout kudu-system-time

  $IMPALA_HOME/bin/bootstrap_development.sh

  source $IMPALA_HOME/bin/impala-config.sh
  (pushd fe && mvn -fae test -Dtest=AnalyzeDDLTest)
  (pushd fe && mvn -fae test -Dtest=AnalyzeKuduDDLTest)

  $IMPALA_HOME/bin/start-impala-cluster.py
  ./tests/run-tests.py query_test/test_kudu.py

Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
---
M bin/bootstrap_system.sh
M docker/entrypoint.sh
M testdata/cluster/admin
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
5 files changed, 22 insertions(+), 57 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/15568/5
-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 2:

Does this change mean that we would no longer need to call ntp-wait before starting the Kudu daemons? see Impala/testdata/cluster/admin

That's been an occasional source of difficulty/flakiness in different testing environments for us, so it would be nice to be able to eliminate it.


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2020 18:20:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] [test] Use `system unsync` time for Kudu test clusters

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 4: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 28 Mar 2020 20:38:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9577: [test] Use `system unsync` time for Kudu test clusters

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15568 )

Change subject: IMPALA-9577: [test] Use `system_unsync` time for Kudu test clusters
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5574/


-- 
To view, visit http://gerrit.cloudera.org:8080/15568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id99e5cb58ab988c3ad4f98484be8db193d5eaf99
Gerrit-Change-Number: 15568
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Mar 2020 02:00:25 +0000
Gerrit-HasComments: No