You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jobs@airflow.apache.org by GitBox <gi...@apache.org> on 2026/04/29 04:25:09 UTC
[GH] (airflow/fix/elasticsearch-compat-with-v2): Workflow run "Tests" failed!
The GitHub Actions job "Tests" on airflow.git/fix/elasticsearch-compat-with-v2 has failed.
Run started by GitHub user petercheon (triggered by eladkal).
Head commit for run:
c0ef998b717743d36cd1ad19a00ecc18cd440d49 / Peter Cheon <pe...@kakaocorp.com>
fix(elasticsearch): pin compatible-with at the transport layer to keep ES 8 servers working
Since #64070 the provider depends on elasticsearch>=8.10,<10. A default
install resolves to an elasticsearch>=9 Python client, which always
negotiates 'compatible-with=9' on every request. Elasticsearch 8.x
servers reject that with HTTP 400 media_type_header_exception, breaking
remote task log ingestion and both ElasticsearchSQLHook and
ElasticsearchPythonHook against ES 8 clusters.
Add a [elasticsearch] es_compat_with config option that, when set to a
major version string ('7'/'8'/'9'), wraps the client's transport
perform_request so every outbound request carries
'Accept: application/vnd.elasticsearch+json; compatible-with=<major>'
(and the matching '+x-ndjson' form for bulk so streaming bodies still
parse). The wrap is applied at every Elasticsearch client construction
site in the provider:
- ElasticsearchTaskHandler (log/es_task_handler.py)
- ElasticsearchRemoteLogIO (log/es_task_handler.py)
- ESConnection (hooks/elasticsearch.py)
- ElasticsearchPythonHook (hooks/elasticsearch.py)
When the option is unset, behavior is unchanged.
Tests assert against what the transport actually sends, not the in-memory
state of the client object. Setting client._headers (which is what
client.options(headers=...) does) is not enough because elasticsearch-py
re-applies its own per-API-method content-negotiation headers right
before the request is sent — only the transport layer sees the final
headers.
Closes: https://github.com/apache/airflow/issues/66063
Supersedes: https://github.com/apache/airflow/pull/66064
Report URL: https://github.com/apache/airflow/actions/runs/25088472994
With regards,
GitHub Actions via GitBox
---------------------------------------------------------------------
To unsubscribe, e-mail: jobs-unsubscribe@airflow.apache.org
For additional commands, e-mail: jobs-help@airflow.apache.org