You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2023/04/24 02:31:00 UTC

[jira] [Updated] (HUDI-2782) Fix marker based strategy for structured streaming

     [ https://issues.apache.org/jira/browse/HUDI-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan updated HUDI-2782:
--------------------------------------
    Fix Version/s:     (was: 0.12.3)

> Fix marker based strategy for structured streaming
> --------------------------------------------------
>
>                 Key: HUDI-2782
>                 URL: https://issues.apache.org/jira/browse/HUDI-2782
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: sivabalan narayanan
>            Priority: Critical
>             Fix For: 0.13.1
>
>
> As part of [this|https://github.com/apache/hudi/pull/3967] patch, we are making timeline server based as the default marker type. But we have an issue w/ structured streaming. Looks like after 1st micro batch, the timeline server gets shutdown and for subsequent micro batches, timeline server is not available. So, in the patch we have made marker based overridden just for structured streaming. 
>  
> We may want to revisit this and see how to go about it. 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)