[GitHub] [hudi] harishraju-govindaraju opened a new issue #5206: How to use DeltaStreamer with AWS Glue

[GitBox <gi...@apache.org>]

harishraju-govindaraju opened a new issue #5206:
URL: https://github.com/apache/hudi/issues/5206


   **_Tips before filing an issue_**
   
   - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
   
   - Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
   
   - If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
   
   **Describe the problem you faced**
   
   DeltaStreamer example with Glue.
   
   **To Reproduce**
   
   I am trying to implement the following design recommendations using Glue instead of EMR.
   
   https://hudi.apache.org/blog/2021/08/23/s3-events-source/
   
   I could find any direct reference on how one can use glue to use deltastreamer libraries. Please point me to some reference or help to provide some sample code samples please. We are using Glue more and not EMR. This would really help.
   
   
   **Expected behavior**
   
   I would like to build micro batch data pipeline using DeltaStreamer with AWS Glue.
   
   **Environment Description**
   
   * Hudi version :
   
   * Spark version :
   
   * Hive version :
   
   * Hadoop version :
   
   * Storage (HDFS/S3/GCS..) :
   
   * Running on Docker? (yes/no) :
   
   
   **Additional context**
   
   Add any other context about the problem here.
   
   **Stacktrace**
   
   ```Add the stacktrace of the error.```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org