[jira] [Commented] (BEAM-1025) User guide - "How to create Beam IO Transforms"

["Stephen Sisk (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891174#comment-15891174 ] 

Stephen Sisk commented on BEAM-1025:
------------------------------------

thanks JB! 

More thinking as I've been looking into what we already have:  Pipeline IO probably has enough content associated with it that we're going to want it to have it's own page (or set of pages.)

I'm going to move in that direction, and try to reconcile the python documentation as we go.

cc [~melap] and [~chamikara] who I believe are also interested in this.

> User guide - "How to create Beam IO Transforms"
> -----------------------------------------------
>
>                 Key: BEAM-1025
>                 URL: https://issues.apache.org/jira/browse/BEAM-1025
>             Project: Beam
>          Issue Type: Task
>          Components: website
>            Reporter: Stephen Sisk
>            Assignee: Stephen Sisk
>
> Beam has javadocs for how to create a read or write transform, but no friendly user guide on how to get started using BoundedSource/BoundedReader.
> This should cover:
> * background on beam's source/sink API design 
> * design patterns
> * evaluating different data sources (eg, what are the properties of a pub sub system that affect how you should write your UnboundedSource? What is the best design for reading from a NoSql style source?)
> * testing - how to write unit, integration (and once we have them, performance tests)
> * public API recommendations
> This is related, but not strictly overlapping with: 
> https://issues.apache.org/jira/browse/BEAM-193
> - the Dataflow SDK documentation for "Custom Sources and Sinks"  contains some info about writing Sources/Sinks, but it is somewhat out of date, and doesn't reflect the things we've learned recently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)