You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/02 12:17:46 UTC
[GitHub] [hudi] schlichtanders commented on issue #6808: [SUPPORT] Cannot sync to spark embedded derby hive meta store (the default one)
schlichtanders commented on issue #6808:
URL: https://github.com/apache/hudi/issues/6808#issuecomment-1264629689
Thank you for your help,
have you tried to replicate it on your side?
I added those configs so that the spark sql returns the following
```
+--------------------------------------------------------------+------------------------------------+
|key |value |
+--------------------------------------------------------------+------------------------------------+
|spark.hadoop.hive.metastore.schema.verification |false |
|spark.hadoop.hive.metastore.schema.verification.record.version|false |
|spark.hadoop.javax.jdo.option.ConnectionDriverName |org.apache.derby.jdbc.EmbeddedDriver|
|spark.hadoop.javax.jdo.option.ConnectionURL |*********(redacted) |
+--------------------------------------------------------------+------------------------------------+
```
Now the code runs through, but no tables get registered. As a test, I am running `df.write.saveAsTable("saveastable_table")` which indeed works.
I won't have the resources to debug this further. I switched to delta lake which works out of the box with sparks local metastore (the one autoenabled by merely setting `spark.sql.catalogImplementation=hive`, or using `SparkSession.builder.enableHiveSupport()` respectively).
Having the possibility to test hudi locally with a local metastore is really crucial for us. It would be great if hudi can support this in the future. An example in the documentation which works would be really great. Best case would be that the default spark metastore works just out of the box (like it is the case for delta lake).
--------------
# Here details about my current try
```python
from pyspark.sql import SparkSession
from pathlib import Path
import os
os.environ["PYSPARK_SUBMIT_ARGS"] = " ".join([
# hudi config
"--packages org.apache.hudi:hudi-spark3.2-bundle_2.12:0.12.0",
"--conf spark.serializer=org.apache.spark.serializer.KryoSerializer",
"--conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog",
"--conf spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension",
# "--conf spark.sql.hive.convertMetastoreParquet=false", # taken from AWS example
# others
# "--conf spark.hadoop.hive.metastore.uris=jdbc:derby:;databaseName=metastore_db;create=true",
# "--conf spark.hadoop.hive.metastore.uris=''",
# f"--conf spark.sql.warehouse.dir={Path('.').absolute() / 'metastore_warehouse'}",
# "--conf spark.eventLog.enabled=false",
"--conf spark.sql.catalogImplementation=hive",
"--conf spark.hadoop.hive.metastore.schema.verification=false",
"--conf spark.hadoop.hive.metastore.schema.verification.record.version=false",
"--conf spark.hadoop.javax.jdo.option.ConnectionDriverName='org.apache.derby.jdbc.EmbeddedDriver'",
"--conf spark.hadoop.javax.jdo.option.ConnectionURL='jdbc:derby:memory:myInMemDB;create=true'",
"--conf spark.hadoop.datanucleus.schema.autoCreateTables=true",
# f"--conf spark.sql.warehouse.dir={Path('.').absolute() / 'metastore_warehouse'}",
# f"--conf spark.sql.hive.metastore.warehouse.dir={Path('.').absolute() / 'metastore_warehouse'}",
# necessary last string
"pyspark-shell",
])
os.environ["PYSPARK_SUBMIT_ARGS"]
spark = SparkSession.builder.getOrCreate()
spark.sql("set").filter("key rlike 'metastore|jdo'").show(1000,False)
sc = spark.sparkContext
sc.setLogLevel("WARN")
dataGen = sc._jvm.org.apache.hudi.QuickstartUtils.DataGenerator()
inserts = sc._jvm.org.apache.hudi.QuickstartUtils.convertToStringList(
dataGen.generateInserts(10)
)
from pyspark.sql.functions import expr
df = spark.read.json(spark.sparkContext.parallelize(inserts, 10)).withColumn(
"part", expr("'foo'")
)
df.toPandas()
tableName = "test_hudi_pyspark_local"
basePath = f"{Path('.').absolute()}/tmp/{tableName}"
hudi_options = {
"hoodie.table.name": tableName,
"hoodie.datasource.write.recordkey.field": "uuid",
"hoodie.datasource.write.partitionpath.field": "part",
"hoodie.datasource.write.table.name": tableName,
"hoodie.datasource.write.operation": "upsert",
"hoodie.datasource.write.precombine.field": "ts",
# "hoodie.upsert.shuffle.parallelism": 2,
# "hoodie.insert.shuffle.parallelism": 2,
"hoodie.datasource.hive_sync.database": "default",
"hoodie.datasource.hive_sync.table": tableName,
"hoodie.datasource.hive_sync.enable": "true",
# "hoodie.datasource.meta.sync.enable": "true",
# "hoodie.datasource.hive_sync.mode": "hiveql",
"hoodie.datasource.hive_sync.mode": "hms",
# "hoodie.datasource.hive_sync.mode": "jdbc",
# "hoodie.datasource.hive_sync.username": "APP",
# "hoodie.datasource.hive_sync.use_jdbc": "false",
# "hoodie.datasource.hive_sync.jdbcurl": f"jdbc:derby:;databaseName={Path('.').absolute() / 'metastore_db'};create=true",
# "hoodie.datasource.hive_sync.jdbcurl": "jdbc:derby:;databaseName=metastore_db;create=true",
"hoodie.datasource.hive_sync.partition_fields": "part",
"hoodie.datasource.hive_sync.partition_extractor_class": "org.apache.hudi.hive.MultiPartKeysValueExtractor",
"index.global.enabled": "true",
"hoodie.index.type": "GLOBAL_BLOOM",
}
(df.write.format("hudi").options(**hudi_options).mode("overwrite").save(basePath))
spark.sql("Show tables from default").toPandas()
df.write.saveAsTable("saveastable_table")
spark.sql("Show tables from default").toPandas()
```
The hoodie write outputs
```
22/10/02 14:12:44 WARN HoodieSparkSqlWriter$: hoodie table at /home/ssahm/Projects_Freelance/Fielmann/bi_kls_data_ocean_import/src/tmp/tmp/test_hudi_pyspark_local already exists. Deleting existing data & overwriting with new data.
22/10/02 14:12:44 WARN HoodieBackedTableMetadata: Metadata table was not found at path /home/ssahm/Projects_Freelance/Fielmann/bi_kls_data_ocean_import/src/tmp/tmp/test_hudi_pyspark_local/.hoodie/metadata
```
and if I don't use `hms` but enable
```
"hoodie.datasource.hive_sync.mode": "jdbc",
"hoodie.datasource.hive_sync.use_jdbc": "true",
```
I even get an error
```
Output exceeds the [size limit](command:workbench.action.openSettings?%5B%22notebook.output.textLineLimit%22%5D). Open the full output data[ in a text editor](command:workbench.action.openLargeOutput?4c0adbbf-d034-45ec-b730-dcdbff52b5cd)
An error occurred while calling o217.save.
: org.apache.hudi.exception.HoodieException: Could not sync using the meta sync class org.apache.hudi.hive.HiveSyncTool
at org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:58)
at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2(HoodieSparkSqlWriter.scala:648)
at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2$adapted(HoodieSparkSqlWriter.scala:647)
at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
at org.apache.hudi.HoodieSparkSqlWriter$.metaSync(HoodieSparkSqlWriter.scala:647)
at org.apache.hudi.HoodieSparkSqlWriter$.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:734)
at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:338)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:183)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:75)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:73)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:84)
at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:110)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:110)
at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:106)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:82)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:481)
...
at java.net.Socket.connect(Socket.java:607)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 67 more
```
```
[0;31m---------------------------------------------------------------------------[0m
[0;31mPy4JJavaError[0m Traceback (most recent call last)
Cell [0;32mIn [16], line 29[0m
[1;32m 2[0m basePath [38;5;241m=[39m [38;5;124mf[39m[38;5;124m"[39m[38;5;132;01m{[39;00mPath([38;5;124m'[39m[38;5;124m.[39m[38;5;124m'[39m)[38;5;241m.[39mabsolute()[38;5;132;01m}[39;00m[38;5;124m/tmp/[39m[38;5;132;01m{[39;00mtableName[38;5;132;01m}[39;00m[38;5;124m"[39m
[1;32m 4[0m hudi_options [38;5;241m=[39m {
[1;32m 5[0m [38;5;124m"[39m[38;5;124mhoodie.table.name[39m[38;5;124m"[39m: tableName,
[1;32m 6[0m [38;5;124m"[39m[38;5;124mhoodie.datasource.write.recordkey.field[39m[38;5;124m"[39m: [38;5;124m"[39m[38;5;124muuid[39m[38;5;124m"[39m,
[0;32m (...)[0m
[1;32m 27[0m [38;5;124m"[39m[38;5;124mhoodie.index.type[39m[38;5;124m"[39m: [38;5;124m"[39m[38;5;124mGLOBAL_BLOOM[39m[38;5;124m"[39m,
[1;32m 28[0m }
[0;32m---> 29[0m ([43mdf[49m[38;5;241;43m.[39;49m[43mwrite[49m[38;5;241;43m.[39;49m[43mformat[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43mhudi[39;49m[38;5;124;43m"[39;49m[43m)[49m[38;5;241;43m.[39;49m[43moptions[49m[43m([49m[38;5;241;43m*[39;49m[38;5;241;43m*[39;49m[43mhudi_options[49m[43m)[49m[38;5;241;43m.[39;49m[43mmode[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43moverwrite[39;49m[38;5;124;43m"[39;49m[43m)[49m[38;5;241;43m.[39;49m[43msave[49m[43m([49m[43mbasePath[49m[43m)[49m)
File [0;32m~/Projects_Freelance/Fielmann/bi_kls_data_ocean_import/src/.venv/lib/python3.9/site-packages/pyspark/sql/readwriter.py:740[0m, in [0;36mDataFrameWriter.save[0;34m(self, path, format, mode, partitionBy, **options)[0m
[1;32m 738[0m [39mself[39m[39m.[39m_jwrite[39m.[39msave()
[1;32m 739[0m [39melse[39;00m:
[0;32m--> 740[0m [39mself[39;49m[39m.[39;49m_jwrite[39m.[39;49msave(path)
File [0;32m~/Projects_Freelance/Fielmann/bi_kls_data_ocean_import/src/.venv/lib/python3.9/site-packages/py4j/java_gateway.py:1321[0m, in [0;36mJavaMember.__call__[0;34m(self, *args)[0m
[1;32m 1315[0m command [39m=[39m proto[39m.[39mCALL_COMMAND_NAME [39m+[39m\
[1;32m 1316[0m [39mself[39m[39m.[39mcommand_header [39m+[39m\
[1;32m 1317[0m args_command [39m+[39m\
[1;32m 1318[0m proto[39m.[39mEND_COMMAND_PART
[1;32m 1320[0m answer [39m=[39m [39mself[39m[39m.[39mgateway_client[39m.[39msend_command(command)
[0;32m-> 1321[0m return_value [39m=[39m get_return_value(
[1;32m 1322[0m answer, [39mself[39;49m[39m.[39;49mgateway_client, [39mself[39;49m[39m.[39;49mtarget_id, [39mself[39;49m[39m.[39;49mname)
[1;32m 1324[0m [39mfor[39;00m temp_arg [39min[39;00m temp_args:
[1;32m 1325[0m temp_arg[39m.[39m_detach()
File [0;32m~/Projects_Freelance/Fielmann/bi_kls_data_ocean_import/src/.venv/lib/python3.9/site-packages/pyspark/sql/utils.py:111[0m, in [0;36mcapture_sql_exception.<locals>.deco[0;34m(*a, **kw)[0m
[1;32m 109[0m [39mdef[39;00m [39mdeco[39m([39m*[39ma, [39m*[39m[39m*[39mkw):
[1;32m 110[0m [39mtry[39;00m:
[0;32m--> 111[0m [39mreturn[39;00m f([39m*[39;49ma, [39m*[39;49m[39m*[39;49mkw)
[1;32m 112[0m [39mexcept[39;00m py4j[39m.[39mprotocol[39m.[39mPy4JJavaError [39mas[39;00m e:
[1;32m 113[0m converted [39m=[39m convert_exception(e[39m.[39mjava_exception)
File [0;32m~/Projects_Freelance/Fielmann/bi_kls_data_ocean_import/src/.venv/lib/python3.9/site-packages/py4j/protocol.py:326[0m, in [0;36mget_return_value[0;34m(answer, gateway_client, target_id, name)[0m
[1;32m 324[0m value [39m=[39m OUTPUT_CONVERTER[[39mtype[39m](answer[[39m2[39m:], gateway_client)
[1;32m 325[0m [39mif[39;00m answer[[39m1[39m] [39m==[39m REFERENCE_TYPE:
[0;32m--> 326[0m [39mraise[39;00m Py4JJavaError(
[1;32m 327[0m [39m"[39m[39mAn error occurred while calling [39m[39m{0}[39;00m[39m{1}[39;00m[39m{2}[39;00m[39m.[39m[39m\n[39;00m[39m"[39m[39m.[39m
[1;32m 328[0m [39mformat[39m(target_id, [39m"[39m[39m.[39m[39m"[39m, name), value)
[1;32m 329[0m [39melse[39;00m:
[1;32m 330[0m [39mraise[39;00m Py4JError(
[1;32m 331[0m [39m"[39m[39mAn error occurred while calling [39m[39m{0}[39;00m[39m{1}[39;00m[39m{2}[39;00m[39m. Trace:[39m[39m\n[39;00m[39m{3}[39;00m[39m\n[39;00m[39m"[39m[39m.[39m
[1;32m 332[0m [39mformat[39m(target_id, [39m"[39m[39m.[39m[39m"[39m, name, value))
[0;31mPy4JJavaError[0m: An error occurred while calling o217.save.
: org.apache.hudi.exception.HoodieException: Could not sync using the meta sync class org.apache.hudi.hive.HiveSyncTool
at org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:58)
at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2(HoodieSparkSqlWriter.scala:648)
at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2$adapted(HoodieSparkSqlWriter.scala:647)
at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
at org.apache.hudi.HoodieSparkSqlWriter$.metaSync(HoodieSparkSqlWriter.scala:647)
at org.apache.hudi.HoodieSparkSqlWriter$.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:734)
at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:338)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:183)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:75)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:73)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:84)
at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:110)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:110)
at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:106)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:481)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:82)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:481)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:457)
at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:106)
at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:93)
at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:91)
at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:128)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:848)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:382)
at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:355)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hudi.exception.HoodieException: Unable to instantiate class org.apache.hudi.hive.HiveSyncTool
at org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:91)
at org.apache.hudi.sync.common.util.SyncUtilHelpers.instantiateMetaSyncTool(SyncUtilHelpers.java:75)
at org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:56)
... 48 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:89)
... 50 more
Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Got runtime exception when hive syncing
at org.apache.hudi.hive.HiveSyncTool.initSyncClient(HiveSyncTool.java:106)
at org.apache.hudi.hive.HiveSyncTool.<init>(HiveSyncTool.java:95)
... 55 more
Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Failed to create HiveMetaStoreClient
at org.apache.hudi.hive.HoodieHiveSyncClient.<init>(HoodieHiveSyncClient.java:95)
at org.apache.hudi.hive.HiveSyncTool.initSyncClient(HiveSyncTool.java:101)
... 56 more
Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Cannot create hive connection jdbc:hive2://localhost:10000/
at org.apache.hudi.hive.ddl.JDBCExecutor.createHiveConnection(JDBCExecutor.java:107)
at org.apache.hudi.hive.ddl.JDBCExecutor.<init>(JDBCExecutor.java:59)
at org.apache.hudi.hive.HoodieHiveSyncClient.<init>(HoodieHiveSyncClient.java:85)
... 57 more
Caused by: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:224)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.hudi.hive.ddl.JDBCExecutor.createHiveConnection(JDBCExecutor.java:104)
... 59 more
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:38)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:311)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:196)
... 63 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 67 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org