You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "PhantomHunt (via GitHub)" <gi...@apache.org> on 2023/08/03 05:54:40 UTC

[GitHub] [hudi] PhantomHunt commented on issue #9344: [SUPPORT] Getting error when writing to different HUDI tables in different threads in same job

PhantomHunt commented on issue #9344:
URL: https://github.com/apache/hudi/issues/9344#issuecomment-1663332820

   We have a Job running on EC2 ubuntu machine that upserts data into 2 hudi tables parallelly in 2 threads (using threadPoolExecutor in the concurrent library of Python) at a time. There are 17 tables in total. When upsertion in any one of the tables is finished, threadPoolExecutor takes in another table to process in the available free thread. The Job terminates when upsertion in all 17 tables finishes. This job runs every 5 mins via cronjob.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org