[jira] [Commented] (BEAM-12388) Improve caching experience on InteractiveRunner with dataframes

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17432617#comment-17432617 ] 

Beam JIRA Bot commented on BEAM-12388:
--------------------------------------

This issue is P2 but has been unassigned without any comment for 60 days so it has been labeled "stale-P2". If this issue is still affecting you, we care! Please comment and remove the label. Otherwise, in 14 days the issue will be moved to P3.

Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed explanation of what these priorities mean.


> Improve caching experience on InteractiveRunner with dataframes
> ---------------------------------------------------------------
>
>                 Key: BEAM-12388
>                 URL: https://issues.apache.org/jira/browse/BEAM-12388
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-py-interactive
>            Reporter: Sam Rohde
>            Priority: P2
>              Labels: stale-P2
>          Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Reusing the default label for to_pcollection when using the interactive runner results in caching errors when used with multiple pipelines:
>  
>  
> {{Traceback (most recent call last):
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py", line 389, in test_dataframes_with_multi_index_get_result
>     pd.testing.assert_series_equal(df_expected, ib.collect(deferred_df, n=10))
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/utils.py", line 247, in run_within_progress_indicator
>     return func(*args, **kwargs)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/interactive_beam.py", line 579, in collect
>     recording = recording_manager.record([pcoll], max_n=n, max_duration=duration)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/recording_manager.py", line 433, in record
>     self._watch(pcolls)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/recording_manager.py", line 306, in _watch
>     for pcoll in to_pcollection(*watched_dataframes, always_return_tuple=True):
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/dataframe/convert.py", line 196, in to_pcollection
>     new_results = {p: extract_input(p)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/transforms/ptransform.py", line 1086, in __ror__
>     return self.transform.__ror__(pvalueish, self.label)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/transforms/ptransform.py", line 587, in __ror__
>     raise ValueError(
> ValueError: Mixing value from different pipelines not allowed.}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)