[GitHub] [beam] allenpradeep commented on pull request #11570: [BEAM-9822] Merge the stages 'Gather and Sort' and 'Create Batches'

[GitBox <gi...@apache.org>]

allenpradeep commented on pull request #11570:
URL: https://github.com/apache/beam/pull/11570#issuecomment-622580239


   This is great niel.  With these changes, there are 3 modes of using SpannerIO write.
   a) Use the conventional way(as it was till now) with a grouping factor where data is grouped, sorted, batched and written as per parameters
   b) Batching without grouping - Set grouping factor as 1 with a larger batched bytes or cells. This will just ensure data is just batched without sort.
   c) No Batching - Set any of the max rows or max mutations or batch bytes to 0 or 1.
   
   Questions:
   1) What mode should our import pipeline use? Should it use option b as data in AVRO seems already sorted?
   2) Where should we document these modes of operation so that some customer can use these?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org