You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/13 18:13:31 UTC
[GitHub] [beam] djaneluz opened a new issue, #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
djaneluz opened a new issue, #21825:
URL: https://github.com/apache/beam/issues/21825
### What happened?
I'm using **RedisIO** to read/write values on cache and it was working fine with version _2.38.0._
When I moved to version _2.39.0_ I started getting errors like:
`Execution of work for computation 'P13' for key '<�֤#�͵.;]�NDC���^m��=�w<@癢P�u:k<��ުm�7����6�';� �z �< z0��>2QN����9EX�-����k\����eL� �( ���a����;��φ' failed with out-of-memory. Work will not be retried locally. Heap dump not written.`
With stacktrace:
`java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:937)
at java.base/java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1583)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:346)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562)
at java.base/java.util.concurrent.Executors$DelegatedScheduledExecutorService.schedule(Executors.java:779)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.OutputAndTimeBoundedSplittableProcessElementInvoker$ProcessContext.onClaimed(OutputAndTimeBoundedSplittableProcessElementInvoker.java:312)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.sdk.fn.splittabledofn.RestrictionTrackers$RestrictionTrackerObserver.tryClaim(RestrictionTrackers.java:60)
at org.apache.beam.sdk.io.redis.RedisIO$ReadFn.processElement(RedisIO.java:399)`
The error happens when calling `RedisIO.readKeyPatterns()`
The pipeline get stuck in the Step: `.../ReadRedis/ParDo(Read)/ParMultiDo(Read)/ProcessKeyedElements.out0`
### Issue Priority
Priority: 0
### Issue Component
Component: io-java-redis
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] djaneluz commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
djaneluz commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1255455551
Hello!
I have a streaming pipeline that consumes Pub/Sub messages, extracts keys, group them by window (FixedWindow of 5 min) and call Redis to get the values, that are used as side input to another step.
Something like:
final PCollection<KV<String, Iterable<MyMessage>>> myMessageKV = pipeline
.apply("ReadPubSubMessages", PubsubIO.readMessages().fromSubscription(options.getSubscription()))
.apply("ExtractAndParse", ParDo.of(new ExtractAndParse()))
.apply("MapMessageWithKey", MapElements.into(TypeDescriptors.kvs(TypeDescriptors.strings(),
TypeDescriptor.of(MyMessage.class)))
.via(m-> KV.of(m.getMyKey(), m)))
.apply("5MinFixedWindow", Window.into(FixedWindows.of(Duration.standardMinutes(5))))
.apply("GroupByKey", GroupByKey.create());
final PCollectionView<Map<String, Iterable<String>>> myCache = myMessageKV
.apply("ExtractKeys", Keys.create())
.apply("ReadRedis", RedisIO.readKeyPatterns()
.withEndpoint(redisHost, REDIS_PORT)
.withOutputParallelization(false))
.apply("ViewAsMultiMapFromCache", View.asMultimap());
final PCollectionTuple outputs = myMessageKV
.apply("EnrichMessage", ParDo.of(new EnrichMessages())
.withSideInput(REF_CODE_CACHE_TAG_ID, refCodeCache));
...
I just ran the pipeline again with BEAM version _2.41.0_ and got the error. With version _2.38.0_ it works just fine.
Let me know what more do you need.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
benWize commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1189332083
Hi Alexey, sorry for the late response, Miguel is busy and he can't take the issue, but we will find someone else in our team, to take this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1194313135
Ping @djaneluz
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1155336237
It can be related to this change #15549 as the latest big change in `RedisIO`.
@MiguelAnzoWizeline @benWize Could you take a look, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
benWize commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1156716602
Hi @aromanenko-dev!
Miguel who made the larger changes in `RedisIO` is no longer working on this project, but I will sync with him and provide a response in a couple of days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] djaneluz commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by "djaneluz (via GitHub)" <gi...@apache.org>.
djaneluz commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1424739514
Hello, is there any update on this? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1156807651
@benWize Many thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1187728484
@benWize Kind ping on this. Do you have any news by chance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] roger-mike commented on issue #21825: [Bug]: RedisIO#readKeyPatterns failing with OutOfMemory on version 2.39.0
Posted by GitBox <gi...@apache.org>.
roger-mike commented on issue #21825:
URL: https://github.com/apache/beam/issues/21825#issuecomment-1194302199
Hi, could you give more details about how to reproduce this error?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org