You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/29 21:52:35 UTC
[GitHub] [airflow] SasanAhmadi opened a new issue #22606: Jenkins JobTriggerOperator bug when polling for new build
SasanAhmadi opened a new issue #22606:
URL: https://github.com/apache/airflow/issues/22606
### Apache Airflow Provider(s)
jenkins
### Versions of Apache Airflow Providers
_No response_
### Apache Airflow version
2.2.3
### Operating System
macOs Monterey 12.0.1
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
No specific details
### What happened
When using the JenkinsJobTriggerOperator there is a polling mechanism to check the built job to return the newly created job number. There is a retry mechanism if it fails to poll it retries couple of times to get the built job number. It is possible though the response when returned does not contain the exact info we are looking for, therefore the part of code which is checking the details of the `executable` key in the body of the jenkins response fail, due to an iteration code over the content of that key which could be None.
It is a faulty code and will result in multiple job creation which could happen in case of a retry on this operator.
```
[2022-03-29, 11:51:40 PDT] {taskinstance.py:1259} INFO - Executing <Task(JenkinsJobTriggerOperator): trigger_downstream_jenkins> on 2022-03-29 17:25:00+00:00
[2022-03-29, 11:51:40 PDT] {standard_task_runner.py:52} INFO - Started process 25 to run task
[2022-03-29, 11:51:40 PDT] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'dag_code', 'trigger_downstream_jenkins', 'scheduled__2022-03-29T17:25:00+00:00', '--job-id', '302242', '--raw', '--subdir', 'DAGS_FOLDER/git_dags/somecode.py', '--cfg-path', '/tmp/tmp84qyueun', '--error-file', '/tmp/tmp732pjeg5']
[2022-03-29, 11:51:40 PDT] {standard_task_runner.py:77} INFO - Job 302242: Subtask trigger_downstream_jenkins
[2022-03-29, 11:51:40 PDT] {logging_mixin.py:109} INFO - Running <TaskInstance: code_v1trigger_downstream_jenkins scheduled__2022-03-29T17:25:00+00:00 [running]> on host codetriggerdownstreamjenkins.79586cc902e641be9e9
[2022-03-29, 11:51:40 PDT] {taskinstance.py:1424} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=data
AIRFLOW_CTX_DAG_ID=Code_v1
AIRFLOW_CTX_TASK_ID=trigger_downstream_jenkins
AIRFLOW_CTX_EXECUTION_DATE=2022-03-29T17:25:00+00:00
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2022-03-29T17:25:00+00:00
[2022-03-29, 11:51:40 PDT] {jenkins_job_trigger.py:182} INFO - Triggering the job/Production/Downstream Trigger - Production on the jenkins : JENKINS with the parameters : None
[2022-03-29, 11:51:40 PDT] {base.py:70} INFO - Using connection to: id: JENKINS. Host: server.com, Port: 443, Schema: , Login: user, Password: ***, extra: True
[2022-03-29, 11:51:40 PDT] {jenkins.py:43} INFO - Trying to connect to [https://server.com:443](https://server.com/)
[2022-03-29, 11:51:40 PDT] {kerberos_.py:325} ERROR - handle_other(): Mutual authentication unavailable on 403 response
[2022-03-29, 11:51:40 PDT] {jenkins_job_trigger.py:160} INFO - Polling jenkins queue at the url https://servercom/queue/item/5831525//api/json
[2022-03-29, 11:51:40 PDT] {taskinstance.py:1700} ERROR - Task failed with exception
Traceback (most recent call last):
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1511, in _execute_task
result = execute_callable(context=context)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 191, in execute
build_number = self.poll_job_in_queue(jenkins_response['headers']['Location'], jenkins_server)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 167, in poll_job_in_queue
if 'executable' in json_response and 'number' in json_response['executable']:
TypeError: argument of type 'NoneType' is not iterable
[2022-03-29, 11:51:40 PDT] {taskinstance.py:1267} INFO - Marking task as UP_FOR_RETRY. dag_id=code_v1 task_id=trigger_downstream_jenkins, execution_date=20220329T172500, start_date=20220329T185140, end_date=20220329T185140
[2022-03-29, 11:51:40 PDT] {standard_task_runner.py:89} ERROR - Failed to execute job 302242 for task trigger_downstream_jenkins
Traceback (most recent call last):
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
args.func(args, dag=self.dag)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 298, in task_run
_run_task_by_selected_method(args, dag, ti)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
_run_raw_task(args, ti)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
ti._run_raw_task(
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1511, in _execute_task
result = execute_callable(context=context)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 191, in execute
build_number = self.poll_job_in_queue(jenkins_response['headers']['Location'], jenkins_server)
File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 167, in poll_job_in_queue
if 'executable' in json_response and 'number' in json_response['executable']:
TypeError: argument of type 'NoneType' is not iterable
[2022-03-29, 11:51:40 PDT] {local_task_job.py:154} INFO - Task exited with return code 1
[2022-03-29, 11:51:40 PDT] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
### What you think should happen instead
There should be an extra check to prevent the iteration over possible None type returned in the `executable` key from the body of the response from jenkins poll request.
This is the current code:
https://github.com/apache/airflow/blob/b0b69f3ea7186e76a04b733022b437b57a087a2e/airflow/providers/jenkins/operators/jenkins_job_trigger.py#L161
It should be updated to this code:
```python
if 'executable' in json_response and json_response['executable'] is not None and 'number' in json_response['executable']:
```
### How to reproduce
During calls to poling to jenkins it is happening randomly, it might not happen at all. It depends on the jenkins performance as well, how fast it could build the jobs might prevent this.
### Anything else
No further info.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #22606: Jenkins JobTriggerOperator bug when polling for new build
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #22606:
URL: https://github.com/apache/airflow/issues/22606#issuecomment-1082408443
feel free to submit PR with a fix
(If you plan to fix a bug you can just open a PR directly and explain the bug on the PR details.. you don't have to open issue first)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org