[GitHub] [airflow] dsynkov commented on pull request #14816: Extend HTTP extra_options to LivyHook and operator

[GitBox <gi...@apache.org>]

dsynkov commented on pull request #14816:
URL: https://github.com/apache/airflow/pull/14816#issuecomment-799872079


   While I was testing this I got the below error:
   ```
   [2021-03-16 01:05:07,136] {logging_mixin.py:104} INFO - Running <TaskInstance: example_livy_operator.pi_java_sensor 2021-03-16T01:05:06.055205+00:00 [running]> on host 0ec62ec2ae59
   [2021-03-16 01:05:07,169] {taskinstance.py:1481} ERROR - Task failed with exception
   Traceback (most recent call last):
     File "/opt/airflow/airflow/models/taskinstance.py", line 1137, in _run_raw_task
       self._prepare_and_execute_task_with_callbacks(context, task)
     File "/opt/airflow/airflow/models/taskinstance.py", line 1274, in _prepare_and_execute_task_with_callbacks
       self.render_templates(context=context)
     File "/opt/airflow/airflow/models/taskinstance.py", line 1776, in render_templates
       self.task.render_template_fields(context)
     File "/opt/airflow/airflow/models/baseoperator.py", line 870, in render_template_fields
       self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
     File "/opt/airflow/airflow/models/baseoperator.py", line 881, in _do_render_template_fields
       content = getattr(parent, attr_name)
   AttributeError: 'LivySensor' object has no attribute 'batch_id'
   ```
   Which looks to be due to `batch_id` in `template_fields` not matching how it's stored in the constructor `self._batch_id` so I fixed this as well.
   
   https://github.com/apache/airflow/blob/956a2053b51d690038f5db6a61ed14f381540240/airflow/providers/apache/livy/sensors/livy.py#L36
   
   https://github.com/apache/airflow/blob/956a2053b51d690038f5db6a61ed14f381540240/airflow/providers/apache/livy/sensors/livy.py#L44


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org