[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6909: [AIRFLOW-6359] spark_submit_hook.py status polling interval config

[GitBox <gi...@apache.org>]

nuclearpinguin commented on a change in pull request #6909: [AIRFLOW-6359] spark_submit_hook.py status polling interval config
URL: https://github.com/apache/airflow/pull/6909#discussion_r361816610
 
 

 ##########
 File path: airflow/contrib/hooks/spark_submit_hook.py
 ##########
 @@ -492,8 +496,11 @@ def _start_driver_status_tracking(self):
         while self._driver_status not in ["FINISHED", "UNKNOWN",
                                           "KILLED", "FAILED", "ERROR"]:
 
-            # Sleep for 1 second as we do not want to spam the cluster
-            time.sleep(1)
+            # Sleep for n seconds as we do not want to spam the cluster
+            if self._status_poll_interval:
 
 Review comment:
   If we use suggestion from init then we can simply do:
   ```suggestion
               time.sleep(self._status_poll_interval)
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services