You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/11/09 15:38:33 UTC

[GitHub] [hudi] Kavin88 commented on issue #3831: Deltastreamer through Pyspark/livy

Kavin88 commented on issue #3831:
URL: https://github.com/apache/hudi/issues/3831#issuecomment-964268810


   i need to ingest 100 tables through deltastreamer and spark job would be running in continuous mode.  My thought is to implement this through python code  with sparksession and do the livy submit calling the pyfile. Could find anywhere to integrate the --props and --source-class parameter directly in python code rather than passing it through spark-submit. Planning to achieve this in generic code and change param values alone for each table. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org