You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "xchwan (via GitHub)" <gi...@apache.org> on 2025/09/22 14:24:29 UTC
[I] Amazon kinesis hook contains filehose class [airflow]
xchwan opened a new issue, #55955:
URL: https://github.com/apache/airflow/issues/55955
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
issue #52712 request to build kinesis provider, which means kinesis data stream.
But, the current kinesis hook in providers/amazon/src/airflow/providers/amazon/aws/hooks/kinesis.py
only has firehose hook.
Base on
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/index.html
boto3 doc. Firehose and Kinesis is not in the same category. The kinesis file should not have filehose hook.
I think the filehose hook should exist in filehose.py independently for kinesis data stream hook set up.
### Apache Airflow version
3
### Operating System
MacOS26
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### What happened
_No response_
### What you think should happen instead
_No response_
### How to reproduce
Only filename need to modify
### Anything else
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3333514601
I think this is more of a question to the AWS team @vincbeck @o-nikolas
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "xchwan (via GitHub)" <gi...@apache.org>.
xchwan commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3326044660
> I would like to help/assist on this. Let me know if help needed
Thanks for your help, but as I said, I’ll make the PR myself. Appreciate it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3332058467
I think it would better to separate to 2 releases.
1. Add Firehouse Hook ( by just import from existed Kinesis Hook ) and add deprecation warning to Kinesis Hook that user should switch to Firehouse Hook.
2. After 1. is released, then replace Kinesis Hook implementation with "Kinesis Data Stream"
cc @eladkal
Any comment for this approach?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3334247066
> I think it would better to separate to 2 releases.
>
> 1. Add Firehouse Hook ( by just import from existed Kinesis Hook ) and add deprecation warning to Kinesis Hook that user should switch to Firehouse Hook.
> 2. After 1. is released, then replace Kinesis Hook implementation with "Kinesis Data Stream"
>
> cc [@eladkal](https://github.com/eladkal)
>
> Any comment for this approach?
Yes and no :) I agree with 1. but I disagree with 2. Doing 2 would means creating a major version for Amazon provider package, but even more importantly having a very short deprecation period for users. We cannot do that. I think we should:
- Do 1 as suggested by @jason810496
- Create Kinesis hook either in `providers/amazon/src/airflow/providers/amazon/aws/hooks/kinesis.py` or `providers/amazon/src/airflow/providers/amazon/aws/hooks/kinesis_analytics.py`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "xchwan (via GitHub)" <gi...@apache.org>.
xchwan closed issue #55955: Amazon kinesis hook contains firehose class
URL: https://github.com/apache/airflow/issues/55955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains filehose class [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3320563388
Sorry - in the issue title that you're created, you've called it "filehouse" (an "l" instead of an "r").
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "xchwan (via GitHub)" <gi...@apache.org>.
xchwan commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3321942752
> Sorry - in the issue title that you're created, you've called it "filehouse" (an "l" instead of an "r").
Modify done. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains filehose class [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3319543732
Thanks for submitting the issue! Can you please update the title to be "firehouse" instead of "filehose"? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains firehose class [airflow]
Posted by "yash1thsa (via GitHub)" <gi...@apache.org>.
yash1thsa commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3325203421
I would like to help/assist on this. Let me know if help needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Amazon kinesis hook contains filehose class [airflow]
Posted by "xchwan (via GitHub)" <gi...@apache.org>.
xchwan commented on issue #55955:
URL: https://github.com/apache/airflow/issues/55955#issuecomment-3319651346
> Thanks for submitting the issue! Can you please update the title to be "firehouse" instead of "filehose"? Please feel free to open up a PR for this issue. Thanks!
<img width="182" height="62" alt="Image" src="https://github.com/user-attachments/assets/27f87c61-409b-44f3-bba8-6c7f4312ceb6" />
It is firehose
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org