You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/13 09:42:35 UTC
[GitHub] [airflow] emincanoguz11 opened a new issue, #27029: Druid Operator is not getting host
emincanoguz11 opened a new issue, #27029:
URL: https://github.com/apache/airflow/issues/27029
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
We use Airflow 2.3.3. I see that this test is successful, but I take a this error. This is the picture
File "/home/airflow/.local/lib/python3.7/site-packages/requests/sessions.py", line 792, in get_adapter raise InvalidSchema(f"No connection adapters were found for {url!r}")
<img width="1756" alt="Screen Shot 2022-10-12 at 15 34 40" src="https://user-images.githubusercontent.com/47830986/195560866-0527c5f6-3795-460b-b78b-2488e2a77bfb.png">
<img width="1685" alt="Screen Shot 2022-10-12 at 15 37 27" src="https://user-images.githubusercontent.com/47830986/195560954-f5604d10-eb7d-4bab-b10b-2684d8fbe4a2.png">
I take dag like this
Also I tried this type but I failed
` ingestion_2 = SimpleHttpOperator(
task_id='test_task',
method='POST',
http_conn_id=DRUID_CONN_ID,
endpoint='/druid/indexer/v1/task',
data=json.dumps(read_file),
dag=dag,
do_xcom_push=True,
headers={
'Content-Type': 'application/json'
},
response_check=lambda response: response.json()['Status'] == 200)
`
I get this log
`[2022-10-13, 06:16:46 UTC] {http.py:143} ERROR - {"error":"Missing type id when trying to resolve subtype of [simple type, class org.apache.druid.indexing.common.task.Task]: missing type id property 'type'\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 1, column: 1]"}
`
I don't know this is bug or issue or networking problem but can we check this?
P.S - We use Airflow on Kubernetes so that we can not debug it.
### What you think should happen instead
_No response_
### How to reproduce
_No response_
### Operating System
Kubernetes
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
o-nikolas commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1278187677
It looks like the druid hook is getting the protocol to use from the conn type, and since you're using a Druid connection that ends up being `druid://` in your url, but `requests` doesn't find any adapters for that protocol:
https://github.com/apache/airflow/blob/0d78ba560dec2e7ea2670744800864906622a4a4/airflow/providers/apache/druid/hooks/druid.py#L65-L69
It also looks like you're not providing an endpoint, the default seems to be `{"endpoint": "druid/indexer/v1/task"}`
I have no Druid experience so I can't advise much more than that unfortunately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279977510
Sorry for removed assigment. It my fault. @o-nikolas can you give assignment to me please again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
o-nikolas commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279991259
> Sorry for removed assigment. It my fault. @o-nikolas can you give assignment to me please again
No worries, done :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #27029: Druid Operator is not getting host
URL: https://github.com/apache/airflow/issues/27029
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1278869081
Hi Nikolas Again,
I tested with standalone Airflow and I tested only http but this is not worked :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279102855
I solved problem. Thanks but this logs are so terrible to defining this problem. Maybe we will create new logs for defining problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1277336935
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279283364
This is ingestion file problem but I see in debug mode for in details. We must see log in Airflow for about ingestion problem. I hope I take PR. Thanks for answering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
o-nikolas commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279369149
> I hope I take PR.
Okay, I've assigned this ticket to you for now :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1278567130
Hi Nikolas,
We saw that this type but I cannot change this. Because we are using Airflow on Kubernetes. But also we gave HTTP Operator to Druid Broker for this reason but it is not worked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1277901526
P.S I updated Airflow 2.4.0 but it is not work :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279976719
Hi Again,
I develop the new log system. First of all we see that
but in my development I see that clearly the main reason of error.
I take PR in few days. Thanks for all kinding supports.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
o-nikolas commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1279236893
Hey @emincanoguz11
> I solved problem. Thanks but this logs are so terrible to defining this problem. Maybe we will create new logs for defining problem.
That sounds great! Can you provide more details on what the actual problem was and what logging you think would improve things? That would be massively helpful to others in the future. You could even open a PR yourself and become one of the many contributors to the airflow project! :rocket:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] emincanoguz11 commented on issue #27029: Druid Operator is not getting host
Posted by GitBox <gi...@apache.org>.
emincanoguz11 commented on issue #27029:
URL: https://github.com/apache/airflow/issues/27029#issuecomment-1286488277
@o-nikolas Hi Nikolas,
If you have a free time please check this PR. Thanks for your support
[PR Link](https://github.com/apache/airflow/pull/27174)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org