[jira] [Commented] (HUDI-3012) Investigate: Metadata table write performance impact

["Manoj Govindassamy (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-3012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468352#comment-17468352 ] 

Manoj Govindassamy commented on HUDI-3012:
------------------------------------------

 

Ran the spark data source based hudi table upserts with and without metadata table and I don't see the performance degradation. The previous test was using integ test suite and the delta streamer. Integ test suite brought in many moving parts for the write and it did bunch of FS based file listings on its own. Closing the issue since i don't see the major performance degradation anymore. 

> Investigate: Metadata table write performance impact
> ----------------------------------------------------
>
>                 Key: HUDI-3012
>                 URL: https://issues.apache.org/jira/browse/HUDI-3012
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: Writer Core
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>            Priority: Blocker
>             Fix For: 0.11.0
>
>
> # Write path: Run Hoodie table inserts/upserts via Spark DataSource or DeltaStreamer and investigate the performance impact
>  # (optional) Read path: Measure the boost on the read side by using the metadata table based file llistings. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)