You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Bill Neubauer (JIRA)" <ji...@apache.org> on 2018/04/05 15:18:03 UTC

[jira] [Created] (BEAM-4017) Go session runner should write multiple files

Bill Neubauer created BEAM-4017:
-----------------------------------

             Summary: Go session runner should write multiple files
                 Key: BEAM-4017
                 URL: https://issues.apache.org/jira/browse/BEAM-4017
             Project: Beam
          Issue Type: Improvement
          Components: sdk-go
            Reporter: Bill Neubauer
            Assignee: Bill Neubauer


The Go session runner allows a worker to "play back" a previous execution, which can be useful for debugging or profiling sessions. However, the recording facility produces one file for the entire lifetime of the worker. While this is useful for local debugging, it won't work well for workers at scale.

Having the session capture facility make the output chunkable will help larger systems scale. I suggest that the interface for session writing be expanded from a io.WriteCloser to include a sequence number that systems can use to produce an ordered set of files for playback.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)