You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "shunping (via GitHub)" <gi...@apache.org> on 2025/12/02 20:02:24 UTC
[I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
shunping opened a new issue, #36965:
URL: https://github.com/apache/beam/issues/36965
### What happened?
This is observed in some of our test workflow. Even though it is not causing the test to fail, I believe it is only because of luck, aka our tests do not cover this case.
Thanks @aIbrahiim for bringing this up in one of the fix PR #36927.
To reproduce, we can use the following code:
```python
import logging
import apache_beam as beam
from apache_beam.coders import VarIntCoder
from apache_beam.transforms.userstate import ReadModifyWriteStateSpec
from apache_beam.utils.timestamp import Timestamp
from apache_beam.options.pipeline_options import PipelineOptions
from apache_beam.testing.test_stream import TestStream
logging.basicConfig(level=logging.INFO)
#logging.basicConfig(level=logging.WARNING)
class MyDoFn(beam.DoFn):
COUNT = ReadModifyWriteStateSpec('count', VarIntCoder())
def __init__(self):
pass
def process(
self,
element,
count_state=beam.DoFn.StateParam(COUNT),
):
print(element)
count = count_state.read()
if not count:
count = 1
else:
count += 1
count_state.write(count)
yield count
options = PipelineOptions([
"--streaming",
"--environment_type=LOOPBACK",
"--runner=PrismRunner",
"--prism_log_kind=dev",
#"--prism_beam_version_override=v2.66.0"
# "--runner=PortableRunner",
# "--job_endpoint=localhost:8073",
])
with beam.Pipeline(options=options) as p:
now = Timestamp.now()
_ = (
p | TestStream().add_elements([(1, 2), (3, 4)])
| 'MyDoFn' >> beam.ParDo(MyDoFn())
| beam.LogElements(
prefix="result=",
level=logging.WARNING,
with_timestamp=True,
with_window=True,
use_epoch_time=True))
```
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #36965:
URL: https://github.com/apache/beam/issues/36965#issuecomment-3993027970
Not yet! Feel free to give it a try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #36965:
URL: https://github.com/apache/beam/issues/36965#issuecomment-5205002597
This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #36965: [Bug]: Prism panic when TestStream is followed by a stateful dofn
URL: https://github.com/apache/beam/issues/36965
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "boy397 (via GitHub)" <gi...@apache.org>.
boy397 commented on issue #36965:
URL: https://github.com/apache/beam/issues/36965#issuecomment-4019667067
Hi @shunping do you have any suggestions??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "boy397 (via GitHub)" <gi...@apache.org>.
boy397 commented on issue #36965:
URL: https://github.com/apache/beam/issues/36965#issuecomment-3993135455
you have worked on it, so is there any suggestion you want to provide me before i start??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "boy397 (via GitHub)" <gi...@apache.org>.
boy397 commented on issue #36965:
URL: https://github.com/apache/beam/issues/36965#issuecomment-3993019445
@shunping Is this issue solved yet? If not can i contribute to it??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Prism panic when TestStream is followed by a stateful dofn [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #36965:
URL: https://github.com/apache/beam/issues/36965#issuecomment-5559276152
This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org