You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "kstokoz (via GitHub)" <gi...@apache.org> on 2026/01/12 12:24:28 UTC
[I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
kstokoz opened a new issue, #2620:
URL: https://github.com/apache/pekko/issues/2620
I was moving my project from pekko 1.0.1 to 1.2.0. One of my test which using source.recoverWithRetries failed.
I believe in change #1775 and #1663 merged to pekko 1.2.0 release in case of Source.failed semantics of the API is actualy changed, i.e. max attempts of -1 is no longer working properly.
Please consider following test:
```
import java.util.concurrent.atomic.AtomicInteger
import org.apache.pekko.stream.scaladsl.Sink
import org.apache.pekko.stream.scaladsl.Source
import org.apache.pekko.util.ByteString
import org.apache.pekko.NotUsed
import scala.concurrent.Await
import scala.concurrent.duration.Duration
class ResilientSourceTest extends UnitTest {
def withRetriesTest(originSource: Source[ByteString, Any])(fallbackTo: Long => Source[ByteString, NotUsed])(shouldRetry: Throwable => Boolean = { _ => true }): Source[ByteString, NotUsed] =
originSource.recoverWithRetries(
-1,
{
case e: Throwable if shouldRetry(e) =>
fallbackTo(0)
}
).mapMaterializedValue(_ => NotUsed)
test("withRetries should stop to retry if shouldRetry decides otherwise") {
val counter = new AtomicInteger()
val source =
withRetriesTest(failedSource("origin")) { _ =>
counter.incrementAndGet()
// failedSource1("works")
failedSource("does not work")
} { _ =>
counter.get() < 3
}
assertThrows[ArithmeticException] {
Await.result(source.runWith(Sink.ignore), Duration.Inf)
}
assert(counter.get() == 3)
}
private def failedSource(message: String): Source[ByteString, NotUsed] =
Source.failed(new ArithmeticException(message))
private def failedSource1(message: String): Source[ByteString, NotUsed] =
Source.single(5).map(_ / 0).map(s => ByteString.fromString(s.toString))
}
```
When i have Source.failed (case failedSource) test is not passing, while i would expect it should, similar if source would produce error during execution (case failedSource1)
Regards,
Kyrylo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin closed issue #2620: RecoverWith operator break of semantics in 1.2.0 release
URL: https://github.com/apache/pekko/issues/2620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3748631687
Whatever gets agreed, we will make sure the docs reflect it.
I'm waiting for other Pekko contributors to voice an opinion before choosing whether to revert the 1.2.0 change or to keep it and document this behaviour change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3752619254
I was in the hospital, will take a look this weekend.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin closed issue #2620: RecoverWith operator break of semantics in 1.2.0 release
URL: https://github.com/apache/pekko/issues/2620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3744414679
I have reproduced the issue. I'm not sure whether we should revert 1775 or try to fix the issue that it introduces with retrying where exceptions occur (`failedSource` in the example).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3758869360
@kstokoz @pjfanning I think we should update the document about this and with a test case in https://github.com/apache/pekko/pull/2624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "kstokoz (via GitHub)" <gi...@apache.org>.
kstokoz commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3744471546
@pjfanning it is indeed a test scenario where i want to test retry logic, having `Source.failed` is defined method to create failing source. Even though it is a test i more worried about sematics change of the API, when failed source created via different options.
I don`t have test for other cases in the change like futureSource or javaSource, but i don`t think assumption that we can shortcut some logic is safe here.
I would vote for reverting the original change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3765330216
@He-Pin @raboof @mdedetrich the more I think about this, the more I think that we should fix this. The recover functions take as one param a function to create a recovery flow. That function could return a hard coded FailedSource in an artificial test but it could have logic that depending on the input value or external factor if the function allows that then it could return a FailedSource or a viable Source. For example first retry could lead to a FailedSource but 2nd could lead to a different type of Source that is worth trying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3766182498
I have changed the implementation and updated the test case, which should fixed this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3738346012
recoverWithRetries is documented to support negative retries and means infinite retries so this looks like a regression (and a bug).
https://pekko.apache.org/docs/pekko/current/stream/operators/Source-or-Flow/recoverWithRetries.html
wdyt @He-Pin ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3744541020
Pekko 1.2.0 is about 4 months old and nobody else has reported issues like this. I would argue that we need a better test case. I'm not fully against a revert but I wouldn't expedite a release with a more worrying scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "He-Pin (via GitHub)" <gi...@apache.org>.
He-Pin commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3766149922
A problem with that is: if the user is not using `recoverWithRetries`, it will retry in an infinite loop.
Even the fix is quite simple.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "kstokoz (via GitHub)" <gi...@apache.org>.
kstokoz commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3748140994
If you think that current behavior is an optimisation and it would keep failing right away on failed source, i think you should document this behavior. I should admit it would very confusing for users that when you set -1 for infinite retries and it doesn`t do it in some cases. Any user who want to provoke error in test or in general will use failed source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3913755376
#2633 and possibly #2631 are implicated in breaking our pekko-http tests so we may need to look at this again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] RecoverWith operator break of semantics in 1.2.0 release [pekko]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
pjfanning commented on issue #2620:
URL: https://github.com/apache/pekko/issues/2620#issuecomment-3745710217
My gut feeling is that https://github.com/apache/pekko/blob/44799471fe6b3fe7dfb9ef49470079be3cc74248/stream/src/main/scala/org/apache/pekko/stream/impl/fusing/Ops.scala#L2183 is safe - that we don't retry if the source is an instance of FailedSource and for me, this seems sensible.
It would be one thing if the source was anything other a FailedSource, a source that had a chance to succeed - but an instanceof FailedSource, it seems to me, that it is an improvement not to retry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org