You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/01 11:56:05 UTC
[GitHub] [airflow] nicolamarangoni opened a new issue, #25442: jenkins_job_trigger requests.exceptions.HTTPError: 400 Client Error: Bad Request for url
nicolamarangoni opened a new issue, #25442:
URL: https://github.com/apache/airflow/issues/25442
### Apache Airflow version
2.3.3 (latest released)
### What happened
The following task using the **JenkinsJobTriggerOperator** to trigger a job on **Jenkins 2.346.1**
```
jenkins_job = JenkinsJobTriggerOperator(
task_id = 'jenkins_job',
jenkins_connection_id = 'jenkins',
job_name = 'PEW/Dev_Pipeline_2',
dag=dag
)
```
runs into the following error since upgrading to 2.3.x:
```
[2022-08-01, 15:00:20 CEST] {jenkins_job_trigger.py:190} INFO - Triggering the job PEW/Dev_Pipeline_2 on the jenkins : jenkins with the parameters : None
[2022-08-01, 15:00:20 CEST] {base.py:68} INFO - Using connection ID 'jenkins' for task execution.
[2022-08-01, 15:00:20 CEST] {warnings.py:109} WARNING - /usr/lib/python3.10/site-packages/airflow/models/connection.py:294: DeprecationWarning: Encountered non-JSON in `extra` field for connection 'jenkins'. Support for non-JSON `extra` will be removed in Airflow 3.0
self._validate_extra(extra_val, self.conn_id)
[2022-08-01, 15:00:20 CEST] {jenkins.py:46} INFO - Trying to connect to [https://jenkins.pharos.pke.fhm.de:443](https://jenkins.pharos.pke.fhm.de/)
[2022-08-01, 15:00:20 CEST] {kerberos_.py:325} ERROR - handle_other(): Mutual authentication unavailable on 403 response
[2022-08-01, 15:00:21 CEST] {taskinstance.py:1909} ERROR - Task failed with exception
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 197, in execute
jenkins_response = self.build_job(jenkins_server, self.parameters)
File "/usr/lib/python3.10/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 133, in build_job
return jenkins_request_with_headers(jenkins_server, request)
File "/usr/lib/python3.10/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py", line 51, in jenkins_request_with_headers
response = jenkins_server.jenkins_request(req)
File "/usr/lib/python3.10/site-packages/jenkins/__init__.py", line 575, in jenkins_request
return self._response_handler(
File "/usr/lib/python3.10/site-packages/jenkins/__init__.py", line 536, in _response_handler
response.raise_for_status()
File "/usr/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: [https://jenkins.pharos.pke.fhm.de:443/job/PEW/job/Dev_Pipeline_2/build](https://jenkins.pharos.pke.fhm.de/job/PEW/job/Dev_Pipeline_2/build)
[2022-08-01, 15:00:21 CEST] {taskinstance.py:1415} INFO - Marking task as FAILED. dag_id=test_start_jenkins_job, task_id=jenkins_job, execution_date=20220801T110017, start_date=20220801T110019, end_date=20220801T110021
[2022-08-01, 15:00:21 CEST] {standard_task_runner.py:92} ERROR - Failed to execute job 97 for task jenkins_job (400 Client Error: Bad Request for url: [https://jenkins.pharos.pke.fhm.de:443/job/PEW/job/Dev_Pipeline_2/build;](https://jenkins.pharos.pke.fhm.de/job/PEW/job/Dev_Pipeline_2/build;) 107)
```
### What you think should happen instead
The **JenkinsJobTriggerOperator** should successfully start a job on Jenkins.
### How to reproduce
Create a DAG with the above Task to trigger a job on **Jenkins 2.346.1**
### Operating System
Alpine Linux 3.16.1
### Versions of Apache Airflow Providers
apache-airflow-providers-apache-hdfs==3.0.1
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.2.0
apache-airflow-providers-common-sql==1.0.0
apache-airflow-providers-datadog==3.0.0
apache-airflow-providers-exasol==2.1.3
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-jenkins==3.0.0
apache-airflow-providers-microsoft-mssql==3.1.0
apache-airflow-providers-odbc==3.1.0
apache-airflow-providers-oracle==3.1.0
apache-airflow-providers-postgres==5.1.0
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-slack==5.1.0
apache-airflow-providers-sqlite==3.1.0
apache-airflow-providers-ssh==3.1.0apache-airflow-providers-apache-hdfs==3.0.1
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.2.0
apache-airflow-providers-common-sql==1.0.0
apache-airflow-providers-datadog==3.0.0
apache-airflow-providers-exasol==2.1.3
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-jenkins==3.0.0
apache-airflow-providers-microsoft-mssql==3.1.0
apache-airflow-providers-odbc==3.1.0
apache-airflow-providers-oracle==3.1.0
apache-airflow-providers-postgres==5.1.0
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-slack==5.1.0
apache-airflow-providers-sqlite==3.1.0
apache-airflow-providers-ssh==3.1.0
### Deployment
Other 3rd-party Helm chart
### Deployment details
One Pod on Kubernetes containing the following containers
- 1 Container for the webserver service
- 1 Container for the scheduler service
- 1 Container for the dag-processor service
- 1 Container for the flower service
- 1 Container for the redis service
- 2 or 3 containers for the celery workers services
Due to a previous issue crashing the scheduler with the message `UNEXPECTED COMMIT - THIS WILL BREAK HA LOCKS`, we substitute `scheduler_job.py` with the file `https://raw.githubusercontent.com/tanelk/airflow/a4b22932e5ac9c2b6f37c8c58345eee0f63cae09/airflow/jobs/scheduler_job.py`.
Sadly I don't remember which issue or MR exactly but it was related to scheduler and dag-processor running in separate containers.
### Anything else
The issue is directly related to the **apache-airflow-providers-jenkins** itself, I could reproduce the issue with the following code:
```
import jenkins, requests
server = jenkins.Jenkins('https://jenkins.pharos.pke.fhm.de', username='UserName', password='myPassword')
user = server.get_whoami()
version = server.get_version()
print('Hello %s from Jenkins %s' % (user['fullName'], version))
request = requests.Request(method='POST', url=server.build_job_url('PEW/Dev_Pipeline_2', None, None))
response = server.jenkins_request(request)
```
This is the result:
```
Hello UserName from Jenkins 2.346.1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.10/site-packages/jenkins/__init__.py", line 575, in jenkins_request
return self._response_handler(
File "/usr/lib/python3.10/site-packages/jenkins/__init__.py", line 536, in _response_handler
response.raise_for_status()
File "/usr/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://jenkins.pharos.pke.fhm.de/job/PEW/job/Dev_Pipeline_2/build
```
It is possible that the libraries **jenkins-python** or **requests** have been broken by the version **2.346.x** of Jenkins, however the Jenkins-provider looks broken.
It looks also like that work of **jenkins-python** is not really active:
https://pypi.org/project/python-jenkins/#history
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #25442: jenkins_job_trigger requests.exceptions.HTTPError: 400 Client Error: Bad Request for url
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #25442:
URL: https://github.com/apache/airflow/issues/25442#issuecomment-1248063813
Cool. Closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #25442: jenkins_job_trigger requests.exceptions.HTTPError: 400 Client Error: Bad Request for url
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #25442:
URL: https://github.com/apache/airflow/issues/25442#issuecomment-1203102177
I hope someone who knows jenkins can take a look and diagnose/fix it. Marked it as good first issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #25442: jenkins_job_trigger requests.exceptions.HTTPError: 400 Client Error: Bad Request for url
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #25442: jenkins_job_trigger requests.exceptions.HTTPError: 400 Client Error: Bad Request for url
URL: https://github.com/apache/airflow/issues/25442
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nicolamarangoni commented on issue #25442: jenkins_job_trigger requests.exceptions.HTTPError: 400 Client Error: Bad Request for url
Posted by GitBox <gi...@apache.org>.
nicolamarangoni commented on issue #25442:
URL: https://github.com/apache/airflow/issues/25442#issuecomment-1247707465
This issue was solved by itself simply rebuilding the image with the current versions of all packages.
It looks like the error was not caused by the python-jenkins package itself but by a dependency (requests? certifi?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org