You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampark.apache.org by "lvheyang (via GitHub)" <gi...@apache.org> on 2025/11/14 09:32:37 UTC

[I] [Feature] StreamPark Support Cloud Serverless Flink [streampark]

lvheyang opened a new issue, #4314:
URL: https://github.com/apache/streampark/issues/4314

   ### Search before asking
   
   - [x] I had searched in the [feature](https://github.com/apache/streampark/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   ### Background
   
   Apache StreamPark is a powerful open-source tool for managing the entire lifecycle of Apache Flink applications. Currently, StreamPark supports deploying Flink jobs to platforms like YARN and Kubernetes. However, with the growing adoption of cloud-native and serverless technologies, many organizations are leveraging fully managed serverless Flink services provided by cloud vendors such as [AWS](https://www.amazonaws.cn/managed-service-apache-flink/pricing/), [Alibaba Cloud](https://help.aliyun.com/zh/flink/), and [Volcengine](https://www.volcengine.com/product/flink) 
   
   ### User Scenarios
   
   **Multi-Cloud Strategy**: Many enterprises operate in multi-cloud environments to avoid vendor lock-in, meet regional compliance requirements, or leverage specific cloud features. StreamPark, as a cloud-agnostic tool, can provide a unified management experience for streaming jobs across different cloud providers.
   
   **Cost and Maintenance Reduction**: Serverless Flink services eliminate the need for infrastructure management, auto-scale based on workload, and reduce operational overhead. Enabling StreamPark to deploy to these services would allow users to focus on development and business logic rather than cluster maintenance.
   
   **Global and Regional Compliance**: Different cloud providers have varying region availability and compliance certifications. By supporting multiple cloud serverless Flink services, StreamPark can help users deploy jobs to the most suitable region or cloud provider based on their business needs.
   
   ### Proposed Solution
   
   Extend StreamPark’s deployment options to include support for cloud serverless Flink services. This would involve:
   
   * Adding new deployment targets for AWS, Alibaba Cloud, Volcengine, and potentially other cloud providers.
   * Implementing provider-specific configurations (e.g., region, IAM roles, service endpoints) in the StreamPark job configuration.
   * Ensuring compatibility with the REST APIs or SDKs provided by these cloud services for job submission and management.
   
   <img width="3296" height="1786" alt="Image" src="https://github.com/user-attachments/assets/51ecc282-1564-4a64-ae58-cabc0a3600d8" />
   
   ### Benefits
   
   * Unified Management: StreamPark users can manage Flink jobs across on-premises, Kubernetes, and multiple cloud environments from a single interface.
   * Reduced Operational Complexity: Leverage serverless Flink offerings to minimize infrastructure management and scaling concerns.
   * Flexibility and Choice: Users can choose the best cloud provider or region for their specific use case without being tied to a single vendor.
   
   We believe this feature would significantly enhance StreamPark’s value proposition for organizations adopting cloud-native and multi-cloud strategies.
   
   ### Usage Scenario
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [SPIP-4][Feature] StreamPark Support Cloud Serverless Flink [streampark]

Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys closed issue #4314: [SPIP-4][Feature] StreamPark Support Cloud Serverless Flink
URL: https://github.com/apache/streampark/issues/4314


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] StreamPark Support Cloud Serverless Flink [streampark]

Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on issue #4314:
URL: https://github.com/apache/streampark/issues/4314#issuecomment-3541131512

   +1 on this feature.
   
   <img width="1457" height="300" alt="Image" src="https://github.com/user-attachments/assets/7164001c-48a6-4a84-90f9-fd788bedef49" />
   
   If needed, you may use a new StreamPark Improve Proposal number, such as SPIP-4.
   
   Looking forward to the feature!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] StreamPark Support Cloud Serverless Flink [streampark]

Posted by "lvheyang (via GitHub)" <gi...@apache.org>.
lvheyang commented on issue #4314:
URL: https://github.com/apache/streampark/issues/4314#issuecomment-3540232414

   Really appreciate the recognition. I'll share my detailed ideas with everyone for review this week.  
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] StreamPark Support Cloud Serverless Flink [streampark]

Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys commented on issue #4314:
URL: https://github.com/apache/streampark/issues/4314#issuecomment-3538844061

   Looks great! Looking forward to the next steps and diving into the details.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [SPIP-4][Feature] StreamPark Support Cloud Serverless Flink [streampark]

Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys commented on issue #4314:
URL: https://github.com/apache/streampark/issues/4314#issuecomment-5522997828

   Thanks for your feedback. We are closing this issue because it is outdated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org