You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "laglangyue (via GitHub)" <gi...@apache.org> on 2024/01/13 07:51:37 UTC
[I] improve usage of Source.combine by adding Source.commbineN(sources: Source[T, _]*) [incubator-pekko]
laglangyue opened a new issue, #932:
URL: https://github.com/apache/incubator-pekko/issues/932
<!--
Please report issues regarding specific projects in their respective issue trackers, e.g.:
- Pekko HTTP: https://github.com/apache/incubator-pekko-http/issues
- Pekko Connectors: https://github.com/apache/incubator-pekko-connectors/issues
- Pekko Persistence Cassandra Plugin: https://github.com/apache/incubator-pekko-persistence-cassandra/issues
- ...
Please explain your use case precisely, and if possible provide an example snippet.
Thanks for contributing!
-->
## motivation
please see this method
```
def mergePrioritizedN[T](
sourcesAndPriorities: immutable.Seq[(Source[T, _], Int)],
eagerComplete: Boolean): Source[T, NotUsed] = {
sourcesAndPriorities match {
case immutable.Seq() => Source.empty
case immutable.Seq((source, _)) => source.mapMaterializedValue(_ => NotUsed)
case sourcesAndPriorities =>
val (sources, priorities) = sourcesAndPriorities.unzip
combine(sources.head, sources(1), sources.drop(2): _*)(_ => MergePrioritized(priorities, eagerComplete))
}
}
```
Users will also write similar methods which is wrapped combine
```
val sources: List[Source[Row, NotUsed]] = parallel.source
val seq = parallel.source.grouped(4).map {
case immutable.List() => Source.empty[Row]
case immutable.List(source) => source
case source => Source.combine(source.head, source(1), source.drop(2)*)(Concat(_))
}.toSeq
// merge
val reader = seq match
case immutable.Seq() => Source.empty
case immutable.Seq(seq) => seq
case seq => Source.combine(seq.head, seq(1), seq.drop(2)*)(Concat(_))
```
original combine must need fisrt and second source.
method `source.mergePrioritizedN` wrappers the method `combine`, perhaps we can promote this method,
but we cannot overload , maybe we can provide a method `SourceN`.
## Additional for case:
data from different databases and tables like shardingSphere，or get tables from MySQL through regular expressions
` list[Source] -> groupd concat -> merge `
Mainly to see what opinions of community
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] improve usage of Source.combine by adding Source.commbineN(sources: Source[T, _]*) [incubator-pekko]
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue commented on issue #932:
URL: https://github.com/apache/incubator-pekko/issues/932#issuecomment-1890369652
it is resoved in 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [I] improve usage of Source.combine by adding Source.commbineN(sources: Source[T, _]*) [incubator-pekko]
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue closed issue #932: improve usage of Source.combine by adding Source.commbineN(sources: Source[T, _]*)
URL: https://github.com/apache/incubator-pekko/issues/932
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org