Spark 3: creating schema for hive metastore hangs forever

[Tomas Bartalos <to...@gmail.com>]

Hello,

I'm using spark-3.0.0-bin-hadoop3.2 with custom hive metastore DB
(postgres). I'm setting the "autoCreateAll" flag to true, so hive is
creating its relational schema on first use. The problem is there is a
deadlock and the query hangs forever:
*Tx1* (*holds lock on TBLS relation*, wait_event: ClientRead):
*INSERT* *INTO* "PARTITION_KEYS" ("TBL_ID","PKEY_COMMENT","PKEY_NAME",
"PKEY_TYPE","INTEGER_IDX") *VALUES* (*$1*,*$2*,*$3*,*$4*,*$5*)
*Tx2 (waiting for lock on TBLS relation, *wait_event: relation*): *
*ALTER* *TABLE* "TBL_PRIVS" *ADD* *CONSTRAINT* "TBL_PRIVS_FK1" *FOREIGN*
*KEY* ("TBL_ID") *REFERENCES* "TBLS" ("TBL_ID") *INITIALLY* *DEFERRED*

So Tx1 waits for client read and Tx2 is waiting for Tx1. I'm attaching the
stack trace of active thread reading from postgres.

Steps to reproduce:
#start postgres
docker run --name postgres-hive -d -p 5432:5432 -e
POSTGRES_PASSWORD=postgres postgres:13

#run shell with hive metastore DB pointing to postgres
./bin/spark-shell \
--packages org.postgresql:postgresql:42.2.14 \
--conf
spark.hadoop.javax.jdo.option.ConnectionURL=jdbc:postgresql://localhost/postgres?createDatabaseIfNotExist=true
\
--conf
spark.hadoop.javax.jdo.option.ConnectionDriverName=org.postgresql.Driver \
--conf spark.hadoop.javax.jdo.option.ConnectionUserName=postgres \
--conf spark.hadoop.javax.jdo.option.ConnectionPassword=postgres \
--conf spark.hadoop.hive.metastore.schema.verification=false \
--conf spark.hadoop.datanucleus.schema.autoCreateAll=true \
--conf spark.hadoop.datanucleus.fixedDatastore=false

#Create any hive table
spark.sql("""
     | CREATE TABLE parquet_test (
     |  id int,
     |  str string,
     |  mp MAP<STRING,STRING>,
     |  lst ARRAY<STRING>,
     |  strct STRUCT<A:STRING,B:STRING>)
     | PARTITIONED BY (part string)
     | STORED AS PARQUET;
     | """)

BR,
Tomas