You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@libcloud.apache.org by GitBox <gi...@apache.org> on 2022/10/14 07:13:37 UTC
[GitHub] [libcloud] rsnk96 opened a new issue, #1786: Supporting Queues / Messaging Pipelines across Cloud Providers
rsnk96 opened a new issue, #1786:
URL: https://github.com/apache/libcloud/issues/1786
## Feature Request
A commonly used feature in Cloud Platforms is their message-passing/queueing services, such as
* [AWS](https://aws.amazon.com/sqs/): SQS
* [Azure](https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted): [Service-Bus, Storage Queues]
* [GCP](https://cloud.google.com/pubsub/docs/choosing-pubsub-or-cloud-tasks): [Cloud Tasks, Cloud Pub-Sub]
These are often used in conjunction with their other services, such as Storage (Ex: to send a message-notification automatically when a new file is uploaded to a Container/Bucket) and Compute (Ex: To pass information between different compute instances, or to take an action whenever a new file is uploaded to Storage). However, it is a pain right now to deal with the different CSP's syntaxes for using these queues, and would be convenient if we could centralize how we carry out at least basic operations like:
1. Read `n` messages from queue
2. Add new messages into the queue
3. Flush the queue / Delete `n` items from the queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] rsnk96 commented on issue #1786: [RFC] Supporting Queues / Messaging Pipelines across Cloud Providers (Ex: AWS SQS)
Posted by "rsnk96 (via GitHub)" <gi...@apache.org>.
rsnk96 commented on issue #1786:
URL: https://github.com/apache/libcloud/issues/1786#issuecomment-1665085982
> Ideally the next step would be a proposal in a form of a PR which implement the base API.
@Kami - can you clarify if you mean the 1st step would be to develop the end-user API, without any backend? (Just to sample the UX)
Also, can you clarify if the IRC Chat (#libcloud) is still the way to go for more immediate support, or would Github Issues / any other mode be the preferred option now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1786: Supporting Queues / Messaging Pipelines across Cloud Providers (Ex: AWS SQS)
Posted by "Kami (via GitHub)" <gi...@apache.org>.
Kami commented on issue #1786:
URL: https://github.com/apache/libcloud/issues/1786#issuecomment-1660342260
Yeah, I think ``libcloud.queue`` / ``libcloud.queing`` API would make sense.
Since queuing APIs are usually quite poll heavy (at least for reading / retrieving messages), I think it would also make sense to expose async API in addition to the sync one.
This would also make it a first abstraction in Libcloud which would native expose async APIs - which makes a lot of sense in this day and age. I know that some library users build their own async primitives on top of other Libcloud APIs which don't expose async methods.
Ideally the next step would be a proposal in a form of a PR which implement the base API. Implementation for at least 2-3 providers is also a good idea to make sure that the base API is generic enough and not biased towards a single provider (that's how we approached other APIs as well - base API + 3 driver implementations).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] rsnk96 commented on issue #1786: Supporting Queues / Messaging Pipelines across Cloud Providers
Posted by GitBox <gi...@apache.org>.
rsnk96 commented on issue #1786:
URL: https://github.com/apache/libcloud/issues/1786#issuecomment-1301732810
Hi @Kami
Can you help me with how to get this feature request reviewed?
(I saw your activity being the most recent, sorry if someone else is better equipped to review this request 😅 )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1786: [RFC] Supporting Queues / Messaging Pipelines across Cloud Providers (Ex: AWS SQS)
Posted by "Kami (via GitHub)" <gi...@apache.org>.
Kami commented on issue #1786:
URL: https://github.com/apache/libcloud/issues/1786#issuecomment-1666901074
@rsnk96 Correct, yeah.
But usually to develop a good and provider agnostic (as much as possible) end user API, it involves prototyping with at least 2-3 providers (to ensure API is not biased against a single provider or similar).
And for now, Github issues should work. When you think you have a proposal / prototype ready, feel free to share a link to it on the mailing list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] rsnk96 commented on issue #1786: [RFC] Supporting Queues / Messaging Pipelines across Cloud Providers (Ex: AWS SQS)
Posted by "rsnk96 (via GitHub)" <gi...@apache.org>.
rsnk96 commented on issue #1786:
URL: https://github.com/apache/libcloud/issues/1786#issuecomment-1661473525
👍 will slowly develop a proposal with that approach in mind.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org