[jira] [Created] (BEAM-4518) Python Game stats example does not work on DirectRunner

["Ahmet Altay (JIRA)" <ji...@apache.org>]

Ahmet Altay created BEAM-4518:
---------------------------------

             Summary: Python Game stats example does not work on DirectRunner
                 Key: BEAM-4518
                 URL: https://issues.apache.org/jira/browse/BEAM-4518
             Project: Beam
          Issue Type: Bug
          Components: sdk-py-core
    Affects Versions: 2.5.0, 2.6.0
            Reporter: Ahmet Altay
             Fix For: 2.6.0


Using the injector and the following command to start the DirectRunner pipeline:

python -m apache_beam.examples.complete.game.game_stats \
--project=google.com:clouddfe \
--topic projects/google.com:clouddfe/topics/leader_board-$USER-topic-1 \
--dataset ${USER}_test --fixed_window_duration 1

Fails with:

ValueError: PCollection of size 2 with more than one element accessed as a singleton view. First two elements encountered are "13.93", "11.7777777778". [while running 'CalculateSpammyUsers/ProcessAndFilter']

Offending code is here:

global_mean_score = (
 sum_scores
 | beam.Values()
 | beam.CombineGlobally(beam.combiners.MeanCombineFn())\
 .as_singleton_view())
 # Filter the user sums using the global mean.
 filtered = (
 sum_scores
 # Use the derived mean total score (global_mean_score) as a side input.
 | 'ProcessAndFilter' >> beam.Filter(
 lambda key_score, global_mean:\
 key_score[1] > global_mean * self.SCORE_WEIGHT,
 global_mean_score))

Since global_mean_score is the result of CombineGlobally, this is either an issue with CombineGlobally or side inputs implementation in DirectRunner. The latter is more likely since it works on DataflowRunner.

cc: [~mariagh]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)