You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2022/01/07 17:40:00 UTC
[jira] (HUDI-1850) Read on table fails if the first write to table failed
[ https://issues.apache.org/jira/browse/HUDI-1850 ]
Raymond Xu deleted comment on HUDI-1850:
----------------------------------
was (Author: githubbot):
pengzhiwei2018 edited a comment on pull request #2903:
URL: https://github.com/apache/hudi/pull/2903#issuecomment-889762744
> This patch needs to be redone a bit. Since w/ sql dml, create relation will be called upfront, the empty table check has to be moved to sql dml layer. I will sync up with @pengzhiwei2018 on how to go about this.
Why should we throw an exception for query empty table? I think return an empty list of rows is more reasonable. When user create table and query the table, it is not friendly to throws an exception. Other data format in spark, like parquet, delta, query empty table also return empty rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Read on table fails if the first write to table failed
> ------------------------------------------------------
>
> Key: HUDI-1850
> URL: https://issues.apache.org/jira/browse/HUDI-1850
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Spark Integration
> Affects Versions: 0.8.0
> Reporter: Vaibhav Sinha
> Priority: Major
> Labels: core-flow-ds, pull-request-available, release-blocker, sev:high, spark
> Fix For: 0.11.0, 0.10.1
>
> Attachments: Screenshot 2021-04-24 at 7.53.22 PM.png
>
>
> {code:java}
> ava.util.NoSuchElementException: No value present in Option
> at org.apache.hudi.common.util.Option.get(Option.java:88) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:215) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchema(TableSchemaResolver.java:166) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchema(TableSchemaResolver.java:155) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.hudi.MergeOnReadSnapshotRelation.<init>(MergeOnReadSnapshotRelation.scala:65) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:99) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:63) ~[hudi-spark3-bundle_2.12-0.8.0.jar:0.8.0]
> at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:354) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
> at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:326) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
> at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:308) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
> at scala.Option.getOrElse(Option.scala:189) ~[scala-library-2.12.10.jar:?]
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:308) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:240) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
> {code}
> The screenshot shows the files that got created before the write had failed.
>
> !Screenshot 2021-04-24 at 7.53.22 PM.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)