[jira] [Commented] (BEAM-12388) Improve caching experience on InteractiveRunner with dataframes

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366746#comment-17366746 ] 

Beam JIRA Bot commented on BEAM-12388:
--------------------------------------

This issue is assigned but has not received an update in 30 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.

> Improve caching experience on InteractiveRunner with dataframes
> ---------------------------------------------------------------
>
>                 Key: BEAM-12388
>                 URL: https://issues.apache.org/jira/browse/BEAM-12388
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-py-interactive
>            Reporter: Sam Rohde
>            Assignee: Sam Rohde
>            Priority: P2
>              Labels: stale-assigned
>
> Reusing the default label for to_pcollection when using the interactive runner results in caching errors when used with multiple pipelines:
>  
>  
> {{Traceback (most recent call last):
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py", line 389, in test_dataframes_with_multi_index_get_result
>     pd.testing.assert_series_equal(df_expected, ib.collect(deferred_df, n=10))
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/utils.py", line 247, in run_within_progress_indicator
>     return func(*args, **kwargs)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/interactive_beam.py", line 579, in collect
>     recording = recording_manager.record([pcoll], max_n=n, max_duration=duration)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/recording_manager.py", line 433, in record
>     self._watch(pcolls)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/runners/interactive/recording_manager.py", line 306, in _watch
>     for pcoll in to_pcollection(*watched_dataframes, always_return_tuple=True):
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/dataframe/convert.py", line 196, in to_pcollection
>     new_results = {p: extract_input(p)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/transforms/ptransform.py", line 1086, in __ror__
>     return self.transform.__ror__(pvalueish, self.label)
>   File "/home/srohde/Workdir/beam/sdks/python/apache_beam/transforms/ptransform.py", line 587, in __ror__
>     raise ValueError(
> ValueError: Mixing value from different pipelines not allowed.}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)