You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "He-Pin (via GitHub)" <gi...@apache.org> on 2025/02/22 08:47:39 UTC
[PR] chore: optmize recoverWith to avoid some materialization [pekko]
He-Pin opened a new pull request, #1775:
URL: https://github.com/apache/pekko/pull/1775
Motivation:
refs： https://github.com/apache/pekko/issues/1663
Result:
Avoid some materialization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: optmize recoverWith to avoid some materialization [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on code in PR #1775:
URL: https://github.com/apache/pekko/pull/1775#discussion_r1966510728
##########
stream/src/main/scala/org/apache/pekko/stream/impl/fusing/Ops.scala:
##########
@@ -2171,14 +2177,29 @@ private[pekko] object TakeWithin {
if (maximumRetries < 0 || attempt < maximumRetries) {
pf.applyOrElse(ex, NotApplied) match {
case _: NotApplied.type => failStage(ex)
- case source: Graph[SourceShape[T] @unchecked, M @unchecked] if TraversalBuilder.isEmptySource(source) =>
- completeStage()
- case other: Graph[SourceShape[T] @unchecked, M @unchecked] =>
- TraversalBuilder.getSingleSource(other) match {
- case OptionVal.Some(singleSource) =>
- emit(out, singleSource.elem.asInstanceOf[T], () => completeStage())
+ case source: Graph[SourceShape[T] @unchecked, M @unchecked] =>
+ TraversalBuilder.getValuePresentedSource(source) match {
+ case OptionVal.Some(graph) => graph match {
+ case singleSource: SingleSource[T @unchecked] => emit(out, singleSource.elem, () => completeStage())
+ case failed: FailedSource[T @unchecked] => failStage(failed.failure)
+ case futureSource: FutureSource[T @unchecked] => futureSource.future.value match {
+ case Some(Success(elem)) => emit(out, elem, () => completeStage())
+ case Some(Failure(ex)) => failStage(ex)
+ case None =>
+ switchTo(source)
+ attempt += 1
+ }
+ case iterableSource: IterableSource[T @unchecked] =>
+ emitMultiple(out, iterableSource.elements, () => completeStage())
+ case javaStreamSource: JavaStreamSource[T @unchecked, _] =>
+ emitMultiple(out, javaStreamSource.open().iterator(), () => completeStage())
+ case maybeEmpty if TraversalBuilder.isEmptySource(maybeEmpty) => completeStage()
Review Comment:
Is this going to be slower than the old code? Old code called `TraversalBuilder.isEmptySource(source)` before calling `TraversalBuilder.getSingleSource`. The new code calls `TraversalBuilder.getValuePresentedSource` before checking `TraversalBuilder.isEmptySource(source)` .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: optmize recoverWith to avoid some materialization [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on code in PR #1775:
URL: https://github.com/apache/pekko/pull/1775#discussion_r1975183481
##########
stream-tests/src/test/scala/org/apache/pekko/stream/scaladsl/FlowRecoverWithSpec.scala:
##########
@@ -23,6 +23,8 @@ import pekko.stream.testkit.StreamSpec
import pekko.stream.testkit.Utils._
import pekko.stream.testkit.scaladsl.TestSink
+import scala.concurrent.Future
Review Comment:
can you keep the scala imports together?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: optimize recoverWith to avoid some materialization [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin merged PR #1775:
URL: https://github.com/apache/pekko/pull/1775
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: optmize recoverWith to avoid some materialization [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on code in PR #1775:
URL: https://github.com/apache/pekko/pull/1775#discussion_r1966687328
##########
stream/src/main/scala/org/apache/pekko/stream/impl/fusing/Ops.scala:
##########
@@ -2171,14 +2177,29 @@ private[pekko] object TakeWithin {
if (maximumRetries < 0 || attempt < maximumRetries) {
pf.applyOrElse(ex, NotApplied) match {
case _: NotApplied.type => failStage(ex)
- case source: Graph[SourceShape[T] @unchecked, M @unchecked] if TraversalBuilder.isEmptySource(source) =>
- completeStage()
- case other: Graph[SourceShape[T] @unchecked, M @unchecked] =>
- TraversalBuilder.getSingleSource(other) match {
- case OptionVal.Some(singleSource) =>
- emit(out, singleSource.elem.asInstanceOf[T], () => completeStage())
+ case source: Graph[SourceShape[T] @unchecked, M @unchecked] =>
+ TraversalBuilder.getValuePresentedSource(source) match {
+ case OptionVal.Some(graph) => graph match {
+ case singleSource: SingleSource[T @unchecked] => emit(out, singleSource.elem, () => completeStage())
+ case failed: FailedSource[T @unchecked] => failStage(failed.failure)
+ case futureSource: FutureSource[T @unchecked] => futureSource.future.value match {
+ case Some(Success(elem)) => emit(out, elem, () => completeStage())
+ case Some(Failure(ex)) => failStage(ex)
+ case None =>
+ switchTo(source)
+ attempt += 1
+ }
+ case iterableSource: IterableSource[T @unchecked] =>
+ emitMultiple(out, iterableSource.elements, () => completeStage())
+ case javaStreamSource: JavaStreamSource[T @unchecked, _] =>
+ emitMultiple(out, javaStreamSource.open().iterator(), () => completeStage())
+ case maybeEmpty if TraversalBuilder.isEmptySource(maybeEmpty) => completeStage()
Review Comment:
I think the empty case should not be that common, or we can remove the `maybeEmpty` test and let it left in a `_ =>` case , but that way, we need to keep the changes in `getValuePresentedSource(source)` in sync with here.
what do you think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: optmize recoverWith to avoid some materialization [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on PR #1775:
URL: https://github.com/apache/pekko/pull/1775#issuecomment-2689850690
@pjfanning @Roiocam I think this one is ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [PR] chore: optmize recoverWith to avoid some materialization [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on PR #1775:
URL: https://github.com/apache/pekko/pull/1775#issuecomment-2690369298
> lgtm
I think we can do a sortimports globally..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org