You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/08/31 14:14:00 UTC
[jira] [Work logged] (BEAM-5250) Python Wordcount fails with Flink
portable streaming
[ https://issues.apache.org/jira/browse/BEAM-5250?focusedWorklogId=140110&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-140110 ]
ASF GitHub Bot logged work on BEAM-5250:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Aug/18 14:13
Start Date: 31/Aug/18 14:13
Worklog Time Spent: 10m
Work Description: mxm opened a new pull request #6315: [BEAM-5250] Fix GlobalWindow's max timestamp in Python SDK
URL: https://github.com/apache/beam/pull/6315
This fixes the Python streaming wordcount failing with an
`IllegalStateException` in the Runner code:
"TimestampCombiner moved element from MAX_TIMESTAMP to earlier time END_OF_GLOBAL_WINDOW"
The Python SDK would create a `GlobalWindow` with a maximum timestamp of
MAX_TIMESTAMP which in turn would shift the 'element hold' in the Runner code
backwards. This is because the Runner's `GlobalWindow` maximum timestamp is
smaller than the Python SDK's.
CC @tweise @angoenka
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) </br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | --- | --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 140110)
Time Spent: 10m
Remaining Estimate: 0h
> Python Wordcount fails with Flink portable streaming
> ----------------------------------------------------
>
> Key: BEAM-5250
> URL: https://issues.apache.org/jira/browse/BEAM-5250
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Thomas Weise
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)