[jira] [Closed] (HUDI-8559) Spark SQL commands fail on AWS EMR

["Danny Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Danny Chen closed HUDI-8559.
----------------------------
    Resolution: Fixed

Fixed via master branch: f6c09cd669014599c16c03544105e4d74bf85e0b

> Spark SQL commands fail on AWS EMR
> ----------------------------------
>
>                 Key: HUDI-8559
>                 URL: https://issues.apache.org/jira/browse/HUDI-8559
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Y Ethan Guo
>            Assignee: Lin Liu
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.0.0
>
>
> With EMR Spark the below error is thrown for any SQL commands with master.
> {code:java}
> java.lang.IllegalStateException: Loop detected; file file:/etc/hudi/conf/hudi-defaults.conf already referenced
>   at org.apache.hudi.common.config.DFSPropertiesConfiguration.addPropsFromFile(DFSPropertiesConfiguration.java:147)
>   at org.apache.hudi.common.config.DFSPropertiesConfiguration.loadGlobalProps(DFSPropertiesConfiguration.java:127)
>   at org.apache.hudi.common.config.DFSPropertiesConfiguration.<clinit>(DFSPropertiesConfiguration.java:71)
>   at org.apache.spark.sql.hudi.ProvidesHoodieConfig$.isSchemaEvolutionEnabled(ProvidesHoodieConfig.scala:594)
>   at org.apache.spark.sql.hudi.Spark34ResolveHudiAlterTableCommand.apply(Spark34ResolveHudiAlterTableCommand.scala:35)
>   at org.apache.spark.sql.hudi.Spark34ResolveHudiAlterTableCommand.apply(Spark34ResolveHudiAlterTableCommand.scala:32)
>   at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:239)
>   at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
>   at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
>   at scala.collection.immutable.List.foldLeft(List.scala:91)
>   at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeBatch$1(RuleExecutor.scala:236) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)