[GitHub] [hudi] 117th commented on issue #4700: [SUPPORT] Adding new column to table is not propagated to Hive via HMS sync mode

[GitBox <gi...@apache.org>]

117th commented on issue #4700:
URL: https://github.com/apache/hudi/issues/4700#issuecomment-1025972707


   @xushiyan I will try with 2.3.3, but it might take some time because devops team have to run it in K8s. I'll come back after getting some results, thnx. Feels like my problem actually hides among version mismatch
   
   @nsivabalan actually I've tried using JDBC but got following error 
   ```
   Caused by: org.apache.spark.sql.AnalysisException: Cannot persist darkstore_inventory_jdbc_hist.supply_surpluses into Hive metastore as table property keys may
    not start with 'spark.sql.': [spark.sql.sources.provider, spark.sql.sources.schema.partCol.0, spark.sql.sources.schema.numParts, spark.sql.sources.schema.numP
   artCols, spark.sql.sources.schema.part.0]
   ```
   We (with colleagues) haven't move forward on JDBC sync on K8s, but AFAIK it actually works with AWS EMR and Hive 3.1.x
   So, we tried to switch to HMS sync on kubernetes and as you see it doesn't as smooth as we previously thought


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org