[GitHub] [accumulo] sjyang18 opened a new issue #2029: Failed to create a new table thru Accumulo Client library (org.apache.accumulo.core.client.AccumuloClient) in Kerberos-enabled Accumulo cluster

[GitBox <gi...@apache.org>]

sjyang18 opened a new issue #2029:
URL: https://github.com/apache/accumulo/issues/2029


   **Describe the bug**
   Accumulo Client should be able to support the same API in both non-kerberos and kerberos enabled accumulo cluster. I was able to use Kerberos key tab and principal to create a table, inject data, drop the table thru Accumulo shell. When I do the same in Java Accumulo Client, my java program may list user, tables, and inject data to an existing table, but failed to create a new table with the following error in client side.
   
   ```
   [main} DEBUG org.apache.accumulo.examples.helloworld.InsertWithKerberos  - Failed to create table
   org.apache.accumulo.core.client.AccumuloException: Internal error processing beginFateOperation
           at org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:402)
           at org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:354)
           at org.apache.accumulo.core.clientImpl.TableOperationsImpl.doTableFateOperation(TableOperationsImpl.java:1692)
           at org.apache.accumulo.core.clientImpl.TableOperationsImpl.create(TableOperationsImpl.java:245)
           at org.apache.accumulo.examples.helloworld.InsertWithKerberos.lambda$main$1(InsertWithKerberos.java:105)
           at java.base/java.security.AccessController.doPrivileged(Native Method)
           at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
           at org.apache.accumulo.examples.helloworld.InsertWithKerberos.main(InsertWithKerberos.java:98)
   Caused by: org.apache.thrift.TApplicationException: Internal error processing beginFateOperation
           at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
           at org.apache.accumulo.core.manager.thrift.FateService$Client.recv_beginFateOperation(FateService.java:91)
           at org.apache.accumulo.core.manager.thrift.FateService$Client.beginFateOperation(FateService.java:77)
           at org.apache.accumulo.core.clientImpl.TableOperationsImpl.beginFateOperation(TableOperationsImpl.java:258)
           at org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:364)
           ... 8 more
   
   ```
   On the manager side, I see the matching error with NPE.
   
   ```
   2021-04-23T00:26:04,450 [thrift.ProcessFunction] ERROR: Internal error processing beginFateOperation
   java.lang.NullPointerException: null
           at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[?:?]
           at org.apache.accumulo.server.rpc.TCredentialsUpdatingInvocationHandler.getTokenClassFromName(TCredentialsUpdatingInvocationHandler.java:158) ~[accumulo-server-base-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at org.apache.accumulo.server.rpc.TCredentialsUpdatingInvocationHandler.updateArgs(TCredentialsUpdatingInvocationHandler.java:95) ~[accumulo-server-base-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at org.apache.accumulo.server.rpc.TCredentialsUpdatingInvocationHandler.invoke(TCredentialsUpdatingInvocationHandler.java:62) ~[accumulo-server-base-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at com.sun.proxy.$Proxy41.beginFateOperation(Unknown Source) ~[?:?]
           at org.apache.accumulo.core.master.thrift.FateService$Processor$beginFateOperation.getResult(FateService.java:421) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at org.apache.accumulo.core.master.thrift.FateService$Processor$beginFateOperation.getResult(FateService.java:400) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) ~[libthrift-0.12.0.jar:0.12.0]
           at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.12.0.jar:0.12.0]
           at org.apache.accumulo.server.rpc.UGIAssumingProcessor.process(UGIAssumingProcessor.java:104) ~[accumulo-server-base-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:63) ~[accumulo-server-base-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
           at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310) ~[libthrift-0.12.0.jar:0.12.0]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
           at java.lang.Thread.run(Thread.java:834) [?:?]
   ```
   
   
   
   **Versions (OS, Maven, Java, and others, as appropriate):**
    - Affected version(s) of this project: [e.g.  Accumulo 2.1.0-SNAPSHOT]
    - OS: [e.g. CentOS 7.5]
    - Others:
   
   **To Reproduce**
   Steps to reproduce the behavior (or a link to an example repository that reproduces the problem):
   1. Start with 'https://github.com/apache/accumulo-examples/blob/main/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java'.
   2. Change code like the following to use kerberos principal and keytab file you have setup for kerberos authentication.
   ```
       try {
   
         final Configuration hadoopConf = new Configuration(false);
         hadoopConf.set(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHENTICATION, "kerberos");
         UserGroupInformation.setConfiguration(hadoopConf);
         UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
   
         final File keytabFile = new File(keytabPath);
         final KerberosToken ktoken = new KerberosToken(principal, keytabFile);
         AccumuloClient aclient = Accumulo.newClient().from(opts.getClientPropsPath())
             .as(principal, ktoken).useSasl().build();
         boolean authenticated = aclient.securityOperations().authenticateUser(principal, ktoken);
         log.debug("AccumuloClient instance created with : " + aclient.whoami() + " for Kerberos, "
             + authenticated);
   
         for (SystemPermission perm : SystemPermission.values()) {
           log.debug("Expected user to have permission: " + perm,
               aclient.securityOperations().hasSystemPermission(aclient.whoami(), perm));
         }
         aclient.tableOperations().create("hellotable");
         aclient.close();
       } catch (Exception e) {
         log.debug("Exception caught in Main....");
         log.debug(e.getMessage());
       }
   ```
   3. Change accumulo-client.properties to have kerberos authentication case. Test environment and configuration with 'accumulo shell' to verify. After verification, drop the test table.
   
   4. Compile the changed code and run the same while you tail the manager log on your head node. You will see the similar failure exceptions I shared above with NPE.
   
   **Expected behavior**
   Accumulo client should be able to create a table in Accumulo. 
   
   **Screenshots**
   If applicable, add screenshots to help explain your problem.
   
   **Additional context**
   I reviewed the Accumulo Shell code and it is also using the same client library, but not sure what additional thing(s) Shell class is doing in order to make the creation work.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org