[flink] branch release-1.13 updated: [FLINK-23936][python] Disable cleanup the Python UDFs which are inactive for a few minutes

[di...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dianfu pushed a commit to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git


The following commit(s) were added to refs/heads/release-1.13 by this push:
     new f11ed5d  [FLINK-23936][python] Disable cleanup the Python UDFs which are inactive for a few minutes
f11ed5d is described below

commit f11ed5de028ffd35900fcf6353a51e7db7a43bd5
Author: Dian Fu <di...@apache.org>
AuthorDate: Tue Aug 24 16:09:59 2021 +0800

    [FLINK-23936][python] Disable cleanup the Python UDFs which are inactive for a few minutes
    
    This would avoid reinitializing the Python UDFs which usually is a heavy operation
    in scenarios such as machine learning, e.g., loading a big ML model
    
    This closes #16958.
---
 flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py b/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py
index e509d2f0..f8c0957 100644
--- a/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py
+++ b/flink-python/pyflink/fn_execution/beam/beam_sdk_worker_main.py
@@ -25,5 +25,9 @@ import pyflink.fn_execution.beam.beam_coders # noqa # pylint: disable=unused-imp
 
 import apache_beam.runners.worker.sdk_worker_main
 
+# disable bundle processor shutdown
+from apache_beam.runners.worker import sdk_worker
+sdk_worker.DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S = 86400 * 365 * 100
+
 if __name__ == '__main__':
     apache_beam.runners.worker.sdk_worker_main.main(sys.argv)