Regarding cube building (EMR HA mode)

[Krishna Bandaru <Kr...@scanbuy.com>]

Hi,

while building a cube in EMR HA mode first I got below error i resolved by attaching aws-glue-datacatalog-hive2-client.jar in Kylin class path ($KYLIN_HOME/lib)
java.lang.RuntimeException: java.io.IOException: MetaException(message:Unable to instantiate a metastore client factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory not found)

then i got below error i resoved by adding joda time jar to kylin class path
> The error: "java.lang.NoSuchMethodError: org.joda.time.format.DateTimeF
> ormatter.withZoneUTC()"

now I am getting below error please suggest how to solve this


com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.auth.internal.AWS4SignerUtils
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2254)
        at com.google.common.cache.LocalCache.get(LocalCache.java:3985)
    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4788)
       at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:315)
  at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:277)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
   at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)
   at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
   at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:94)
    at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:122)
        at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:100)
    at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:131)
  at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
    at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
      at org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:111)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.auth.internal.AWS4SignerUtils
  at com.amazonaws.auth.internal.AWS4SignerRequestParams.<init>(AWS4SignerRequestParams.java:99)
       at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:225)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1271)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
  at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
     at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
  at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
      at com.amazonaws.services.glue.AWSGlueClient.doInvoke(AWSGlueClient.java:6396)
   at com.amazonaws.services.glue.AWSGlueClient.invoke(AWSGlueClient.java:6363)
  at com.amazonaws.services.glue.AWSGlueClient.invoke(AWSGlueClient.java:6352)
 at com.amazonaws.services.glue.AWSGlueClient.executeGetDatabase(AWSGlueClient.java:3049)
    at com.amazonaws.services.glue.AWSGlueClient.getDatabase(AWSGlueClient.java:3020)
      at com.amazonaws.glue.catalog.metastore.AWSCatalogMetastoreClient.doesDefaultDBExist(AWSCatalogMetastoreClient.java:247)
 at com.amazonaws.glue.catalog.metastore.AWSCatalogMetastoreClient.<init>(AWSCatalogMetastoreClient.java:165)
        at com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory.createMetaStoreClient(AWSGlueDataCatalogHiveClientFactory.java:19)
     at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:507)
       at org.apache.hive.hcatalog.common.HiveClientCache.getNonCachedHiveMetastoreClient(HiveClientCache.java:99)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:318)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:315)
       at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4791)
       at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3584)
      at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2372)
     at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2335)
     at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2250)
 ... 20 more

Thanks
krishna