You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2022/03/30 04:55:00 UTC
[jira] [Commented] (HUDI-2893) Metadata table not found warn logs repeated
[ https://issues.apache.org/jira/browse/HUDI-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17514439#comment-17514439 ]
Ethan Guo commented on HUDI-2893:
---------------------------------
I confirmed by running the spark quick start guide that this is no longer a problem:
{code:java}
scala> df.write.format("hudi").
| options(getQuickstartWriteConfigs).
| option(PRECOMBINE_FIELD_OPT_KEY, "ts").
| option(RECORDKEY_FIELD_OPT_KEY, "uuid").
| option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
| option(TABLE_NAME, tableName).
| mode(Overwrite).
| save(basePath)
warning: one deprecation; for details, enable `:setting -deprecation' or `:replay -deprecation'
22/03/29 21:53:04 WARN DFSPropertiesConfiguration: Cannot find HUDI_CONF_DIR, please set it as the dir of hudi-defaults.conf
22/03/29 21:53:04 WARN DFSPropertiesConfiguration: Properties file file:/etc/hudi/conf/hudi-defaults.conf not found. Ignoring to load props file
22/03/29 21:53:05 WARN HoodieBackedTableMetadata: Metadata table was not found at path file:///tmp/hudi_trips_cow/.hoodie/metadata
22/03/29 21:53:06 WARN MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties {code}
> Metadata table not found warn logs repeated
> --------------------------------------------
>
> Key: HUDI-2893
> URL: https://issues.apache.org/jira/browse/HUDI-2893
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: sivabalan narayanan
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.11.0
>
>
> When a new table is created for first time, we see repeated warn log statments that metadata table is not found. We need to trim this down to just once.
> // steps to reproduce: just run our quick start guide inserts.
> {code:java}
> scala> df.write.format("hudi").
> | options(getQuickstartWriteConfigs).
> | option(PRECOMBINE_FIELD.key(), "ts").
> | option(PARTITIONPATH_FIELD.key(), "partitionpath").
> | option(TBL_NAME.key(), tableName).
> | mode(Overwrite).
> | save(basePath)
> 21/11/30 11:12:20 WARN DFSPropertiesConfiguration: Cannot find HUDI_CONF_DIR, please set it as the dir of hudi-defaults.conf
> 21/11/30 11:12:20 WARN HoodieSparkSqlWriter$: hoodie table at file:/tmp/hudi_trips_cow already exists. Deleting existing data & overwriting with new data.
> 21/11/30 11:12:21 WARN HoodieBackedTableMetadata: Metadata table was not found at path file:///tmp/hudi_trips_cow/.hoodie/metadata
> 21/11/30 11:12:21 WARN HoodieBackedTableMetadata: Metadata table was not found at path file:///tmp/hudi_trips_cow/.hoodie/metadata
> 21/11/30 11:12:26 WARN HoodieBackedTableMetadata: Metadata table was not found at path file:///tmp/hudi_trips_cow/.hoodie/metadata
> 21/11/30 11:12:26 WARN HoodieBackedTableMetadata: Metadata table was not found at path file:///tmp/hudi_trips_cow/.hoodie/metadata {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)