[GitHub] [beam] kkucharc commented on a change in pull request #12151: [BEAM-9896] Add streaming for SnowflakeIO.Write to Java SDK

[GitBox <gi...@apache.org>]

kkucharc commented on a change in pull request #12151:
URL: https://github.com/apache/beam/pull/12151#discussion_r453721108



##########
File path: sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeIO.java
##########
@@ -719,7 +906,9 @@ private void checkArguments() {
                           return getUserDataMapper().mapRow(element);
                         }
                       }))
-              .apply("Map Objects array to CSV lines", ParDo.of(new MapObjectsArrayToCsvFn()))
+              .apply(
+                  "Map Objects array to CSV lines",
+                  ParDo.of(new MapObjectsArrayToCsvFn(getQuotationMark())))

Review comment:
       Yes, I agree that CSV can be a little bit problematic here. On the other hand apparently it is the most efficient way of loading data to Snowflake.
   We have some tests that were checking different ways of data quotation and problematic characters. But I agree it should be included in [integration tests](https://github.com/apache/beam/blob/master/sdks/java/io/snowflake/src/test/java/org/apache/beam/sdk/io/snowflake/test/BatchSnowflakeIOIT.java) as well. Would it be ok with you if I create Jira ticket for extending integration tests?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org