You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Hennadiy Leontyev (JIRA)" <ji...@apache.org> on 2019/01/03 18:08:00 UTC

[jira] [Created] (BEAM-6356) Python FileBasedCacheManager does not respect PCoder for PCollection being cached

Hennadiy Leontyev created BEAM-6356:
---------------------------------------

             Summary: Python  FileBasedCacheManager does not respect PCoder for PCollection being cached
                 Key: BEAM-6356
                 URL: https://issues.apache.org/jira/browse/BEAM-6356
             Project: Beam
          Issue Type: Improvement
          Components: examples-python
            Reporter: Hennadiy Leontyev
            Assignee: Ahmet Altay


FileBasedCacheManager used by Python's InteractiveRunner does not preserve PCoder for elements of a PCollection being cached on disk. I suggest that the cache on-disk format to be changed to TFRecords (which are supported by Beam) and FileBasedCacheManager would store the desired PCoder for cached collections.
Currently, it is not possible to work with dynamically-generated protocol buffer messages in interactive runner mode because of pickling errors.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)