[GitHub] [airflow] KevinYang21 commented on a change in pull request #5177: [AIRFLOW-4084] Fix bug downloading incomplete logs from ElasticSearch

[GitBox <gi...@apache.org>]

KevinYang21 commented on a change in pull request #5177: [AIRFLOW-4084] Fix bug downloading incomplete logs from ElasticSearch
URL: https://github.com/apache/airflow/pull/5177#discussion_r283560947
 
 

 ##########
 File path: airflow/utils/log/es_task_handler.py
 ##########
 @@ -139,9 +140,17 @@ def es_read(self, log_id, offset):
             .sort('offset')
 
         s = s.filter('range', offset={'gt': offset})
+        max_log_line = s.count()
+        if 'download_logs' in metadata and metadata['download_logs'] and 'max_offset' not in metadata:
+            try:
+                metadata['max_offset'] = s[max_log_line - 1].execute()[-1].offset if max_log_line > 0 else 0
+            except Exception as e:
+                msg = 'Could not get current log size with log_id: {}, ' \
 
 Review comment:
   think we can just pass e to exception and log the msg separately if you want to keep it

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services