You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "mdedetrich (via GitHub)" <gi...@apache.org> on 2023/12/23 23:32:27 UTC
[I] Add a way to create a `Source` from a `Sink` to Akka Streams DSL [incubator-pekko]
mdedetrich opened a new issue, #865:
URL: https://github.com/apache/incubator-pekko/issues/865
This has probably been top of the list when it comes to dealing with commonplace limitations when using pekko-streams, which is the fact that its not easy to create a `Source` when a `Sink` completes/materializes. While it may be true that that a `Sink` should always be the final step in your stream logic there is some impedance mismatch here as there are cases where this props up in reality all the time.
As an example, imagine if you are using the pekko-http [`Flow` API](https://nightlies.apache.org/pekko/docs/pekko-http/1.0/docs/client-side/request-level.html#flow-based-variant) where you represent your entire route as a pekko stream. Lets say your route does a simple thing where it just pushes a message to Kafka and you only want to return a response when sending the message is successful (as is customary).
Understandably the act of sending a message to a [Kafka is a Sink](https://pekko.apache.org/docs/pekko-connectors-kafka/current/producer.html#producer-as-a-sink) which is where the problem lies, once you send the message to Kafka thats the end of your stream business logic and in this specific case of using the pekko-http flow API, we have to get from a `Sink[ProduceRecord[_,_],_]` to a `Source[HttpResponse, _]` in order to return a response to the Http request.
It is currently possible to get around this by using techniques such as materializing the sink and then creating a `Source` from that using `Sink.fromSubscriber`/`Source.fromPublisher` but this creates its own issues as only [cancellations are propagated, not errors](https://github.com/akka/akka/issues/24853#issuecomment-1525680152) so its ideal to just create a proper API for this, i.e. `Source.fromSink(...)` or something along these lines.
Existing Akka issues/discussions on this topic. Do note that all actual code contributions on this topic have been done by external contributors (i.e. no one from Lightbend/Akka) and the code never ended up being merged.
- https://github.com/akka/akka/issues/24853
- https://github.com/akka/akka/pull/25150
- https://discuss.lightbend.com/t/create-source-from-sink-and-vice-versa/605/3
@He-Pin Not sure if you want to look into this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] Add a way to create a `Source` from a `Sink` to Akka Streams DSL [incubator-pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on issue #865:
URL: https://github.com/apache/incubator-pekko/issues/865#issuecomment-1868447512
@mdedetrich What about the `BroadcastHub` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org