You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/01/04 03:27:25 UTC

[GitHub] [hudi] quitozang opened a new issue #2401: [SUPPORT] error queue for deltastreamer

quitozang opened a new issue #2401:
URL: https://github.com/apache/hudi/issues/2401


   When using DeltaStreamer for data storage, if there is abnormal data in Kafka, the DeltaStreamer task will fail to execute. Maybe this is not friendly for a resident pipeline task like DeltaStreamer in continuous mode. 
   Is it possible to add an error queue to store abnormal data?
   
   **Environment Description**
   
   * Hudi version : 0.6.0
   
   * Spark version : 2.4.4
   
   * Hadoop version : 3.2.1
   
   * Storage (HDFS/S3/GCS..) : HDFS
   
   * Running on Docker? (yes/no) : no
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] bvaradar commented on issue #2401: [SUPPORT] error queue for deltastreamer

Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #2401:
URL: https://github.com/apache/hudi/issues/2401#issuecomment-754442329


   Added a jira to track this new dev feature : https://issues.apache.org/jira/browse/HUDI-1505


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] bvaradar closed issue #2401: [SUPPORT] error queue for deltastreamer

Posted by GitBox <gi...@apache.org>.
bvaradar closed issue #2401:
URL: https://github.com/apache/hudi/issues/2401


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] bvaradar closed issue #2401: [SUPPORT] error queue for deltastreamer

Posted by GitBox <gi...@apache.org>.
bvaradar closed issue #2401:
URL: https://github.com/apache/hudi/issues/2401


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org