[GitHub] [airflow] gpoulin commented on issue #23760: OOME when using GCSObjectsWithPrefixExistenceSensor on bucket with many objects

[GitBox <gi...@apache.org>]

gpoulin commented on issue #23760:
URL: https://github.com/apache/airflow/issues/23760#issuecomment-1137478854

   @potiuk how do you usually manage backward compatibility? GCSHook.list already take a `max_result` parameter, but it's actually the pagination size [ref](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/hooks/gcs.py#L673). If I was to write that from scratch I would have 2 parameters: max_result (number of returned values) and pagination_size (number of result fetch at once). However, making this change will break backward compatibility. The alternative is adding a parameter like `max_returned_result` but it would be confusing to have both `max_result` and `max_returned_result`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org