You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Attila Jeges (Code Review)" <ge...@cloudera.org> on 2020/01/30 12:42:30 UTC

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support 

Attila Jeges has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15134


Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................

IMPALA-9279: Update the Kudu version to include VARCHAR support

Before this change the preferred way of getting Kudu was to pull
it in from the specified CDH build (even if USE_CDP_HIVE was set
to true). Optionally by setting USE_CDH_KUDU to false, one could
force Impala to use the native toolchain Kudu. But even then, the
Kudu Java artifacts would be downloaded from CDH.

Since Kudu VARCHAR support won't be backported to CDH, this
behavior blocks the Impala side of the Kudu/Impala VARCHAR
integration.

With this change:
1. Using the native toolchain Kudu (including the Java artifacts)
   is the default behavior. From now on USE_CDH_KUDU will be set
   to false by default. Impala can be forced to fall back on
   using the CDH Kudu by explicitly setting USE_CDH_KUDU to true.
2. Kudu version is updated to include the VARCHAR support.

Testing:
Ran exhaustive tests with USE_CDH_KUDU=true and
USE_CDH_KUDU=false.

Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M impala-parent/pom.xml
3 files changed, 42 insertions(+), 26 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/15134/1
-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/5198/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 14:14:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5548/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jan 2020 13:41:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................

IMPALA-9279: Update the Kudu version to include VARCHAR support

Before this change the preferred way of getting Kudu was to pull
it in from the specified CDH build (even if USE_CDP_HIVE was set
to true). Optionally by setting USE_CDH_KUDU to false, one could
force Impala to use the native toolchain Kudu. But even then, the
Kudu Java artifacts would be downloaded from CDH.

Since Kudu VARCHAR support won't be backported to CDH, this
behavior blocks the Impala side of the Kudu/Impala VARCHAR
integration.

With this change:
1. Using the native toolchain Kudu (including the Java artifacts)
   is the default behavior. From now on USE_CDH_KUDU will be set
   to false by default. Impala can be forced to fall back on
   using the CDH Kudu by explicitly setting USE_CDH_KUDU to true.
2. Kudu version is updated to include the VARCHAR support.

Testing:
Ran exhaustive tests with USE_CDH_KUDU=true and
USE_CDH_KUDU=false.

Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M impala-parent/pom.xml
3 files changed, 42 insertions(+), 25 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/15134/8
-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5477/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jan 2020 20:04:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5300/


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Feb 2020 14:17:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

> > Patch Set 2:
 > > The verify job failed because kudu-3ba5ec5d0 (kudu-1.12.0-SNAPSHOT)
 > has a new run-time dependency: libcurl.so.4 which is not available
 > in the ubuntu-16.04-configured jenkins worker label. I'm discussing
 > with laszlog the possibility of adding libcurls.so.4 to the worker
 > labe;.
 > >
 > 
 > If we decide to take this new Kudu version as a dependency, then
 > the correct way to handle libcurl.so.4 as a new runtime dependency
 > is to add it to the list of packages we install in
 > bin/bootstrap_system.sh.
 > The worker image referenced above is only minimally preconfigured
 > to allow fast startup times; Impala runtime/development time
 > dependencies should be managed in the bootstrap scripts.
 > 
 > Additionally, the dependency on libcurl.so.4 should be evaluated
 > for all OS platforms we claim to have support for: e.g. a brief
 > scan of this article[1] claims that running both libcurl.so.3 and
 > libcurl.so.4 on Ubuntu 18.04 is at least non-trivial to set up.
 > 
 > [1]: https://dev.to/jake/using-libcurl3-and-libcurl4-on-ubuntu-1804-bionic-184g,
 > "Using libcurl3 and libcurl4 on Ubuntu 18.04 (Bionic)"

In bin/bootstrap_system.sh, I don't see us installing curl for ubuntu, but I see us installing it for centos. I would try adding it and see if that helps. (We have curl installed in all the docker images we use to build kudu for the native toolchain.)

We can run a ubuntu-18.04-from-scratch job to see if it works.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jan 2020 18:03:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 6:

I just saw this commit in Kudu, and I'm wondering if it helps with the libcurl situation: https://gerrit.cloudera.org/#/c/15180/

If they link libcurl, then we might not need to install it.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Sat, 08 Feb 2020 01:42:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 6:

> I just saw this commit in Kudu, and I'm wondering if it helps with
 > the libcurl situation: https://gerrit.cloudera.org/#/c/15180/
 > 
 > If they link libcurl, then we might not need to install it.

Correct, this Kudu change eliminates the runtime dependency on the libcurl shared library. 

Here's a native-toolchain CR to bump kudu version once again to include the fix:
https://gerrit.cloudera.org/#/c/15192/


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 10 Feb 2020 15:43:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5321/


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 23:20:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

> Uploaded patch set 7.

Bumped Kudu version and got rid of installing libcurl3 dependency.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 13:30:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5302/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Feb 2020 14:20:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5547/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jan 2020 13:27:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5477/


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jan 2020 21:41:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................

IMPALA-9279: Update the Kudu version to include VARCHAR support

Before this change the preferred way of getting Kudu was to pull
it in from the specified CDH build (even if USE_CDP_HIVE was set
to true). Optionally by setting USE_CDH_KUDU to false, one could
force Impala to use the native toolchain Kudu. But even then, the
Kudu Java artifacts would be downloaded from CDH.

Since Kudu VARCHAR support won't be backported to CDH, this
behavior blocks the Impala side of the Kudu/Impala VARCHAR
integration.

With this change:
1. Using the native toolchain Kudu (including the Java artifacts)
   is the default behavior. From now on USE_CDH_KUDU will be set
   to false by default. Impala can be forced to fall back on
   using the CDH Kudu by explicitly setting USE_CDH_KUDU to true.
2. Kudu version is updated to include the VARCHAR support.

Testing:
Ran exhaustive tests with USE_CDH_KUDU=true and
USE_CDH_KUDU=false.

Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M impala-parent/pom.xml
3 files changed, 43 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/15134/2
-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5493/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 21:40:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 9: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 9
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Feb 2020 13:27:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 9: Code-Review+2

Carry +2


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 9
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Feb 2020 08:33:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

(2 comments)

> > > Patch Set 2:
 > > > The verify job failed because kudu-3ba5ec5d0 (kudu-1.12.0-SNAPSHOT)
 > > has a new run-time dependency: libcurl.so.4 which is not
 > available
 > > in the ubuntu-16.04-configured jenkins worker label. I'm
 > discussing
 > > with laszlog the possibility of adding libcurls.so.4 to the
 > worker
 > > labe;.
 > > >
 > >
 > > If we decide to take this new Kudu version as a dependency, then
 > > the correct way to handle libcurl.so.4 as a new runtime
 > dependency
 > > is to add it to the list of packages we install in
 > > bin/bootstrap_system.sh.
 > > The worker image referenced above is only minimally preconfigured
 > > to allow fast startup times; Impala runtime/development time
 > > dependencies should be managed in the bootstrap scripts.
 > >
 > > Additionally, the dependency on libcurl.so.4 should be evaluated
 > > for all OS platforms we claim to have support for: e.g. a brief
 > > scan of this article[1] claims that running both libcurl.so.3 and
 > > libcurl.so.4 on Ubuntu 18.04 is at least non-trivial to set up.
 > >
 > > [1]: https://dev.to/jake/using-libcurl3-and-libcurl4-on-ubuntu-1804-bionic-184g,
 > > "Using libcurl3 and libcurl4 on Ubuntu 18.04 (Bionic)"
 > 
 > In bin/bootstrap_system.sh, I don't see us installing curl for
 > ubuntu, but I see us installing it for centos. I would try adding
 > it and see if that helps. (We have curl installed in all the docker
 > images we use to build kudu for the native toolchain.)
 > 
 > We can run a ubuntu-18.04-from-scratch job to see if it works.

Installing curl on Ubuntu 16.04 installs libcurl-gnutls.so.4 but it doesn't install the required libcurl.so.4.

"apt install libcurl3" on the other hand works for all supported Ubuntu releases, so I've added that to bin/bootstrap_system.sh.

http://gerrit.cloudera.org:8080/#/c/15134/2/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/15134/2/bin/impala-config.sh@719
PS2, Line 719:   export IMPALA_TOOLCHAIN_KUDU_MAVEN_REPOSITORY="file://${IMPALA_TOOLCHAIN}"
> Since this is disabled, I think we can set it to an empty string. If that w
Setting url to an empty string results in an error but I can set it to something like "file:///non/existing/repo" 

What do you think?


http://gerrit.cloudera.org:8080/#/c/15134/2/bin/impala-config.sh@722
PS2, Line 722:   export IMPALA_KUDU_VERSION="3ba5ec5d0"
             :   export IMPALA_KUDU_JAVA_VERSION="1.12.0-SNAPSHOT"
> One use case that we want to support is for someone to be able to override 
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 14:49:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

> Uploaded patch set 8.

Rebased patch-set.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 13:29:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5323/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 9
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Feb 2020 08:34:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5318/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 13:31:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

> (2 comments)
 > 
 > Thanks for working on this. This is looking pretty good. I'm
 > thinking through the edge cases where we want to override some
 > versions, so I may have a couple more comments.
 > 
 > In the meantime, I'm going to run an upstream verify job on this
 > review.

The verify job failed because kudu-3ba5ec5d0 (kudu-1.12.0-SNAPSHOT) has a new run-time dependency: libcurl.so.4 which is not available in the ubuntu-16.04-configured jenkins worker label. I'm discussing with laszlog the possibility of adding libcurls.so.4 to the worker labe;.

As far as I know, both CDH GBN Kudu and CDP GBN Kudu are based on kudu-1.11.0 which doesn't depend on libcurl.so.4. I considered updating toolchain Kudu to 1.11.0 or 1.11.1 (which is the latest upstream Kudu release) instead of 3ba5ec5d0, but kudu-1.11.x doesn't have support for VARCHAR yet.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jan 2020 14:11:20 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5491/


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 19:42:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................

IMPALA-9279: Update the Kudu version to include VARCHAR support

Before this change the preferred way of getting Kudu was to pull
it in from the specified CDH build (even if USE_CDP_HIVE was set
to true). Optionally by setting USE_CDH_KUDU to false, one could
force Impala to use the native toolchain Kudu. But even then, the
Kudu Java artifacts would be downloaded from CDH.

Since Kudu VARCHAR support won't be backported to CDH, this
behavior blocks the Impala side of the Kudu/Impala VARCHAR
integration.

With this change:
1. Using the native toolchain Kudu (including the Java artifacts)
   is the default behavior. From now on USE_CDH_KUDU will be set
   to false by default. Impala can be forced to fall back on
   using the CDH Kudu by explicitly setting USE_CDH_KUDU to true.
2. Kudu version is updated to include the VARCHAR support.

Testing:
Ran exhaustive tests with USE_CDH_KUDU=true and
USE_CDH_KUDU=false.

Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M impala-parent/pom.xml
3 files changed, 42 insertions(+), 25 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/15134/7
-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8: Code-Review+2

Thanks for bringing in that newer Kudu version to get rid of the curl dependency. That was my only concern here.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 18:36:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Feb 2020 19:15:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5199/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 15:13:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5491/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 14:55:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5300/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Feb 2020 09:27:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

> Patch Set 2:
> The verify job failed because kudu-3ba5ec5d0 (kudu-1.12.0-SNAPSHOT) has a new run-time dependency: libcurl.so.4 which is not available in the ubuntu-16.04-configured jenkins worker label. I'm discussing with laszlog the possibility of adding libcurls.so.4 to the worker labe;.
> 

If we decide to take this new Kudu version as a dependency, then the correct way to handle libcurl.so.4 as a new runtime dependency is to add it to the list of packages we install in bin/bootstrap_system.sh.
The worker image referenced above is only minimally preconfigured to allow fast startup times; Impala runtime/development time dependencies should be managed in the bootstrap scripts.

Additionally, the dependency on libcurl.so.4 should be evaluated for all OS platforms we claim to have support for: e.g. a brief scan of this article[1] claims that running both libcurl.so.3 and libcurl.so.4 on Ubuntu 18.04 is at least non-trivial to set up.

[1]: https://dev.to/jake/using-libcurl3-and-libcurl4-on-ubuntu-1804-bionic-184g, "Using libcurl3 and libcurl4 on Ubuntu 18.04 (Bionic)"


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jan 2020 17:12:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5318/


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 18:22:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

(2 comments)

Thanks for working on this. This is looking pretty good. I'm thinking through the edge cases where we want to override some versions, so I may have a couple more comments.

In the meantime, I'm going to run an upstream verify job on this review.

http://gerrit.cloudera.org:8080/#/c/15134/2/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/15134/2/bin/impala-config.sh@719
PS2, Line 719:   export IMPALA_TOOLCHAIN_KUDU_MAVEN_REPOSITORY="file://${IMPALA_TOOLCHAIN}"
Since this is disabled, I think we can set it to an empty string. If that works, use it.


http://gerrit.cloudera.org:8080/#/c/15134/2/bin/impala-config.sh@722
PS2, Line 722:   export IMPALA_KUDU_VERSION="3ba5ec5d0"
             :   export IMPALA_KUDU_JAVA_VERSION="1.12.0-SNAPSHOT"
One use case that we want to support is for someone to be able to override the IMPALA_KUDU_VERSION and IMPALA_KUDU_JAVA_VERSION and build against a custom Kudu. One might export these variables in bin/impala-config-branch.sh.

So, IMPALA_KUDU_VERSION and IMPALA_KUDU_JAVA_VERSION should respect a preset if it is set. i.e.
IMPALA_KUDU_VERSION=${IMPALA_KUDU_VERSION-"3ba5ec5d0"}
IMPALA_KUDU_JAVA_VERSION=${IMPALA_KUDU_JAVA_VERSION-"1.12.0-SNAPSHOT"}



-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jan 2020 20:04:12 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................

IMPALA-9279: Update the Kudu version to include VARCHAR support

Before this change the preferred way of getting Kudu was to pull
it in from the specified CDH build (even if USE_CDP_HIVE was set
to true). Optionally by setting USE_CDH_KUDU to false, one could
force Impala to use the native toolchain Kudu. But even then, the
Kudu Java artifacts would be downloaded from CDH.

Since Kudu VARCHAR support won't be backported to CDH, this
behavior blocks the Impala side of the Kudu/Impala VARCHAR
integration.

With this change:
1. Using the native toolchain Kudu (including the Java artifacts)
   is the default behavior. From now on USE_CDH_KUDU will be set
   to false by default. Impala can be forced to fall back on
   using the CDH Kudu by explicitly setting USE_CDH_KUDU to true.
2. Kudu version is updated to include the VARCHAR support.

Testing:
Ran exhaustive tests with USE_CDH_KUDU=true and
USE_CDH_KUDU=false.

Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Reviewed-on: http://gerrit.cloudera.org:8080/15134
Reviewed-by: Attila Jeges <at...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M impala-parent/pom.xml
3 files changed, 42 insertions(+), 25 deletions(-)

Approvals:
  Attila Jeges: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 10
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Attila Jeges (Code Review)" <ge...@cloudera.org>.
Attila Jeges has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................

IMPALA-9279: Update the Kudu version to include VARCHAR support

Before this change the preferred way of getting Kudu was to pull
it in from the specified CDH build (even if USE_CDP_HIVE was set
to true). Optionally by setting USE_CDH_KUDU to false, one could
force Impala to use the native toolchain Kudu. But even then, the
Kudu Java artifacts would be downloaded from CDH.

Since Kudu VARCHAR support won't be backported to CDH, this
behavior blocks the Impala side of the Kudu/Impala VARCHAR
integration.

With this change:
1. Using the native toolchain Kudu (including the Java artifacts)
   is the default behavior. From now on USE_CDH_KUDU will be set
   to false by default. Impala can be forced to fall back on
   using the CDH Kudu by explicitly setting USE_CDH_KUDU to true.
2. Kudu version is updated to include the VARCHAR support.

Testing:
Ran exhaustive tests with USE_CDH_KUDU=true and
USE_CDH_KUDU=false.

Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
---
M bin/bootstrap_system.sh
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M impala-parent/pom.xml
4 files changed, 43 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/15134/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 2:

[reply thread elided for brevity's sake]
I have hijacked an unused (for now) build label, adding the GNU-TLS flavor of libcurl.so in the instance startup script. I started 2 builds there:
1. Attila's change (patch set #2) with core mode tests: this passed at https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9901/
2. Core mode tests on master to see if adding libcurl.so.4 leads to any regression. This test passed as well, see: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/9896/

I think the main question here is if we want/need to pick up a pre-release snapshot of Kudu.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jan 2020 21:25:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5321/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 18:31:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5591/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 15:34:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5197/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Feb 2020 14:08:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Feb 2020 02:26:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9279: Update the Kudu version to include VARCHAR support

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15134 )

Change subject: IMPALA-9279: Update the Kudu version to include VARCHAR support
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15134/1/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/15134/1/bin/impala-config.sh@726
PS1, Line 726:   export IMPALA_TOOLCHAIN_KUDU_MAVEN_REPOSITORY="file://${IMPALA_KUDU_JAVA_HOME}/repository"
line too long (92 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/15134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe56342d43cb63e35c0bbb1b4a99327dda0a44a
Gerrit-Change-Number: 15134
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jan 2020 12:43:06 +0000
Gerrit-HasComments: Yes