You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Ferenc Szabo (Code Review)" <ge...@cloudera.org> on 2018/08/27 05:18:38 UTC
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Ferenc Szabo has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11334
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a priviligedExecuter action
Added an extra step to the mini cluster to create
a keyTab for the client used for testing
Added automated test with short KDC ticket lifetime
to test reacquiring
Manual testing was done on a secure cluster as well
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
A java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
7 files changed, 259 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/1
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 1
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Attila Bukor, Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11334
to look at the new patch set (#5).
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a PrivilegedExecutor action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KeyedKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/RegexpKuduOperationsProducerTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
12 files changed, 365 insertions(+), 254 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/5
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 5
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 3:
(8 comments)
This looks great. I just have a couple of questions and some pretty minor nitpicky feedback.
http://gerrit.cloudera.org:8080/#/c/11334/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11334/3//COMMIT_MSG@10
PS3, Line 10: priviligedExecuter
nit: PrivilegedExecutor
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
File java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java@137
PS3, Line 137: // Creating client with FlumeAuthenticator
nit: end regular comment sentence w/ a period.
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java@217
PS3, Line 217:
nit: indent at least 4 spaces on a line continuation
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java@144
PS3, Line 144:
nit: extra indentation?
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java@54
PS3, Line 54: context.put(KERBEROS_KEYTAB, clusterRoot + "/krb5kdc/test-user.keytab");
: context.put(KERBEROS_PRINCIPAL, "test-user@KRBTEST.COM");
Are there some kinds of prerequisites we should document in the javadoc for this method for these values to work? Or is this just how the Kudu MiniKdc sets everything up?
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java@61
PS3, Line 61: KuduClient syncClient, Context context, String tableName, List<Event> events
: ) throws EventDeliveryException {
nit: please use 4-space indentation on both continuation lines, like this:
static void processEventsCreatingSink(
KuduClient syncClient, Context context, String tableName, List<Event> events)
throws EventDeliveryException {
KuduSink sink = ...
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@51
PS3, Line 51: private static final int TICKET_LIFETIME = 10;
: private static final int RENEWABLE_LIFETIME = 30;
nit: append _SEC or _SECONDS to these variable names
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@57
PS3, Line 57: System.clearProperty(KUDU_TICKETCACHE_PROPERTY);
Could this affect other Kudu client tests? Should we be restoring this when the test ends? I am not sure what kind of JVM isolation we get with surefire.
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 3
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Fri, 21 Sep 2018 00:59:29 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Attila Bukor, Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11334
to look at the new patch set (#6).
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a PrivilegedExecutor action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KeyedKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/RegexpKuduOperationsProducerTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
12 files changed, 365 insertions(+), 254 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/6
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 6
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 2:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
File java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java@71
PS1, Line 71: public static final String KERBEROS_KEYTAB = "kerberosKeytab";
> Currently, these are just another property of another sink. There is no cen
It could be worth unifying these across all of flume. Of course that doesn't need to be done for this patch.
http://gerrit.cloudera.org:8080/#/c/11334/2/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/2/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@68
PS2, Line 68: public void setUpBase() throws Exception {
Instead of overriding setUpBase, I think you can create your own setUp method that just handles the System.clearProperty. Setting the kdc configuration can be done by overriding getMiniClusterBuilder.
Here is an example: https://github.com/apache/kudu/blob/c727118320b03fc87befd9941841cdddf8b30762/java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurityContextRealUser.java#L38-L60
http://gerrit.cloudera.org:8080/#/c/11334/2/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@92
PS2, Line 92: public void testEventsWithShosrTickets() throws Exception {
typo: Shosr
http://gerrit.cloudera.org:8080/#/c/11334/2/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@123
PS2, Line 123: private void processEvents(KuduSink sink, int from, int to) throws EventDeliveryException {
Can this be move to a static utility test class and used in KuduSinkTest too?
http://gerrit.cloudera.org:8080/#/c/11334/2/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@145
PS2, Line 145: private KuduSink createSink(String tableName) {
This method is the same very similar KuduSinkTest.createSink. Can it be move to a static utility test class? You could call this createSecureSkink which calls to the createSink from KuduSinkTest.
http://gerrit.cloudera.org:8080/#/c/11334/2/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@162
PS2, Line 162: private KuduTable createNewTable(String tableName) throws Exception {
This method is the same as KuduSinkTest.createNewTable. Can it be move to a static utility test class?
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 2
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Tue, 04 Sep 2018 14:53:24 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 1:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/11334/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11334/1//COMMIT_MSG@9
PS1, Line 9: Adding FlumeAuthenticator to KuduSink and creating KuduClient
Nit. Period at the end of sentences.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
File java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java@71
PS1, Line 71: public static final String KERBEROS_KEYTAB = "kerberosKeytab";
Are these Kudu sink specific or can they be imported from somewhere else in flume?
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java
File java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java@22
PS1, Line 22: * This class is needed because the {@link FlumeAuthenticationUtil#clearCredentials()}
Can a change be made in Flume to change the visibility? Either public or protected could work. Or is it possible to extend FlumeAuthenticatorUtilTestAdapter and add a new public method that way?
I would rather not have org.apache.flume structure here.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java@57
PS1, Line 57: protected KuduTable createNewTable(String tableName) throws Exception {
Perhaps move these to a new test utility class instead of marking them protected.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@90
PS1, Line 90: public void testEventsWithShosrTickets() throws Exception {
Typo in method name.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@124
PS1, Line 124: tx = channel.getTransaction();
This section of code (line 124 through 140) is used in this test and KuduSinkTests a few times, could we break it out into a test utility method? Or maybe enhance testEventsWithDefaults to handle this test too. Perhaps the utility method could take a pre-made sink and then do everything else testEventsWithDefaults does.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@154
PS1, Line 154: protected KuduSink createSink(String tableName, Context ctx) {
This could go in the utility class too. It could be called createSecureSink and pass the kerberos parameters via the Context down to the createSink from KuduSinkTest. These methods are exactly the same otherwise.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@161
PS1, Line 161: parameters.put(KuduSinkConfigurationConstants.KERBEROS_KEYTAB, clusterRoot + "/krb5kdc/flume-client.keytab");
Can we use an existing user keytab and principle so we don't need to create a flume specific one in the mini-cluster?
http://gerrit.cloudera.org:8080/#/c/11334/1/src/kudu/mini-cluster/external_mini_cluster.cc
File src/kudu/mini-cluster/external_mini_cluster.cc:
http://gerrit.cloudera.org:8080/#/c/11334/1/src/kudu/mini-cluster/external_mini_cluster.cc@184
PS1, Line 184: RETURN_NOT_OK_PREPEND(kdc_->CreateServiceKeytab("flume-client", &ktpath),
Can we use the test-user instead of a service keytab so this isn't needed?
http://gerrit.cloudera.org:8080/#/c/11334/1/src/kudu/mini-cluster/external_mini_cluster.cc@734
PS1, Line 734: "--user_acl=test-user,flume-client",
Can we use test-user instead of flume-client so this isn't needed?
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 1
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Mon, 27 Aug 2018 15:14:40 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 3
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Tue, 11 Sep 2018 13:55:04 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Attila Bukor, Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11334
to look at the new patch set (#4).
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a PrivilegedExecutor action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KeyedKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/RegexpKuduOperationsProducerTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
12 files changed, 365 insertions(+), 254 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/4
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 4
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a PrivilegedExecutor action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Reviewed-on: http://gerrit.cloudera.org:8080/11334
Tested-by: Kudu Jenkins
Reviewed-by: Mike Percy <mp...@apache.org>
---
M java/kudu-client/src/test/java/org/apache/kudu/client/BaseKuduTest.java
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KeyedKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/RegexpKuduOperationsProducerTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
13 files changed, 372 insertions(+), 254 deletions(-)
Approvals:
Kudu Jenkins: Verified
Mike Percy: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 8
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 7
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Fri, 28 Sep 2018 18:43:31 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Ferenc Szabo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 2:
Grant, could you help me identify the cause of build failure?
The same test passed locally.
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 2
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Mon, 03 Sep 2018 09:49:45 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Ferenc Szabo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
File java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java@217
PS3, Line 217:
> nit: indent at least 4 spaces on a line continuation
Checkstyle did not mention this indentation error.
You might need to fine-tune the checkstyle config.
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java@54
PS3, Line 54: context.put(KERBEROS_KEYTAB, clusterRoot + "/krb5kdc/test-user.keytab");
: context.put(KERBEROS_PRINCIPAL, "test-user@KRBTEST.COM");
> Are there some kinds of prerequisites we should document in the javadoc for
This user is created by the mini cluster. I have only added the creation of the keytab file.
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/3/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java@57
PS3, Line 57: System.clearProperty(KUDU_TICKETCACHE_PROPERTY);
> Could this affect other Kudu client tests? Should we be restoring this when
The current config is to fork jvm for every test class, so this does not affect the other test. The mini cluster sets it in the begining of the tests as well. It would be restored by the minicluster even without jvm isolation in the @Before of the BaseKuduTest
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 3
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Comment-Date: Wed, 26 Sep 2018 10:14:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Ferenc Szabo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 1:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
File java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java@71
PS1, Line 71: public static final String KERBEROS_KEYTAB = "kerberosKeytab";
> Are these Kudu sink specific or can they be imported from somewhere else in
Currently, these are just another property of another sink. There is no centralized/modularized version of these configuration constants. So in a way, it is Kudu sink specific, however, it is a repeating concept in flume.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java
File java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/flume/auth/FlumeAuthenticatorUtilTestAdapter.java@22
PS1, Line 22: * This class is needed because the {@link FlumeAuthenticationUtil#clearCredentials()}
> Can a change be made in Flume to change the visibility? Either public or pr
Any change on the Flume side would take a long time to release and get back here.
I could use reflection to call it from a more convenient test util.
Another solution would be to have only one test method in one or two Secure test classes and that way we do not have to reset the FlumeAuthenticationUtil.
The basic functionality does not change if it is on a secure cluster so it does not make too much sense anyway.
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
File java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java:
http://gerrit.cloudera.org:8080/#/c/11334/1/java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java@57
PS1, Line 57: protected KuduTable createNewTable(String tableName) throws Exception {
> Perhaps move these to a new test utility class instead of marking them prot
I will do a refactor/cleanup with the rest of the fixes
http://gerrit.cloudera.org:8080/#/c/11334/1/src/kudu/mini-cluster/external_mini_cluster.cc
File src/kudu/mini-cluster/external_mini_cluster.cc:
http://gerrit.cloudera.org:8080/#/c/11334/1/src/kudu/mini-cluster/external_mini_cluster.cc@184
PS1, Line 184: RETURN_NOT_OK_PREPEND(kdc_->CreateServiceKeytab("flume-client", &ktpath),
> Can we use the test-user instead of a service keytab so this isn't needed?
The point of the test is that flume uses a keytab to authenticate.
I could use the test-user and create a new method to generate a keytab, as the one we have now changes the password for the test-user and that breaks a few test cases.
should I go with the new method (CreateKeytabForExistingPrincipal)?
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 1
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Mon, 27 Aug 2018 19:11:31 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11334
to look at the new patch set (#2).
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a priviligedExecuter action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
7 files changed, 241 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/2
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 2
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Hello Attila Bukor, Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11334
to look at the new patch set (#3).
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a priviligedExecuter action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KeyedKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/RegexpKuduOperationsProducerTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
12 files changed, 364 insertions(+), 253 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/3
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 3
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/11334 )
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
Patch Set 2:
> Patch Set 2:
>
> Grant, could you help me identify the cause of build failure?
> The same test passed locally.
I believe the failures was unrelated to your patch
failed to download task files: WARNING 146 isolateserver(1484): Adding unknown file 8878e85ac24180b48ffe02058011a40682f0dc7b to cache
WARNING 152 isolateserver(1490): Added back 1 unknown files
INFO 165 tools(106): Profiling: Section Setup took 0.092 seconds
INFO 194 tools(106): Profiling: Section GetIsolateds took 0.028 seconds
INFO 196 tools(106): Profiling: Section GetRest took 0.002 seconds
INFO 210 isolateserver(1365): 1 ( 105315kb) added
INFO 211 isolateserver(1369): 3446 ( 2152873kb) current
INFO 211 isolateserver(1372): 0 ( 0kb) removed
INFO 211 isolateserver(1375): 45757444kb free
INFO 211 tools(106): Profiling: Section CleanupTrimming took 0.015 seconds
INFO 212 isolateserver(1365): 1 ( 105315kb) added
INFO 213 isolateserver(1369): 3446 ( 2152873kb) current
INFO 213 isolateserver(1372): 0 ( 0kb) removed
INFO 213 isolateserver(1375): 45757444kb free
INFO 213 tools(106): Profiling: Section CleanupTrimming took 0.002 seconds
INFO 213 isolateserver(381): Waiting for all threads to die...
INFO 213 isolateserver(390): Done.
Traceback (most recent call last):
File "/swarming.client/isolateserver.py", line 2211, in <module>
sys.exit(main(sys.argv[1:]))
File "/swarming.client/isolateserver.py", line 2204, in main
return dispatcher.execute(OptionParserIsolateServer(), args)
File "/swarming.client/third_party/depot_tools/subcommand.py", line 242, in execute
return command(parser, args[1:])
File "/swarming.client/isolateserver.py", line 2064, in CMDdownload
require_command=False)
File "/swarming.client/isolateserver.py", line 1827, in fetch_isolated
create_directories(outdir, bundle.files)
File "/swarming.client/isolateserver.py", line 212, in create_directories
os.mkdir(os.path.join(base_directory, d))
OSError: [Errno 17] File exists: '/tmp/dist-test-task12wpkc/build'
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 2
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Tue, 04 Sep 2018 15:37:45 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2012 Kudu Flume sink auth support
Posted by "Ferenc Szabo (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy, Attila Bukor, Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11334
to look at the new patch set (#7).
Change subject: KUDU-2012 Kudu Flume sink auth support
......................................................................
KUDU-2012 Kudu Flume sink auth support
Adding FlumeAuthenticator to KuduSink and creating KuduClient
inside a PrivilegedExecutor action.
Added an extra step to the mini cluster to create
a keyTab for the client used for testing.
Added automated test with short KDC ticket lifetime
to test reacquiring.
Manual testing was done on a secure cluster as well.
Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
---
M java/kudu-client/src/test/java/org/apache/kudu/client/BaseKuduTest.java
M java/kudu-client/src/test/java/org/apache/kudu/client/MiniKuduCluster.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/AvroKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KeyedKuduOperationsProducerTest.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/KuduSinkTestUtil.java
M java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/RegexpKuduOperationsProducerTest.java
A java/kudu-flume-sink/src/test/java/org/apache/kudu/flume/sink/SecureKuduSinkTest.java
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/security/test/mini_kdc.cc
M src/kudu/security/test/mini_kdc.h
13 files changed, 372 insertions(+), 254 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/11334/7
--
To view, visit http://gerrit.cloudera.org:8080/11334
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11b5f08802883afa178d346af48d3bcd15281917
Gerrit-Change-Number: 11334
Gerrit-PatchSet: 7
Gerrit-Owner: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Ferenc Szabo <fs...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>