[GitHub] [nifi] pvillard31 commented on a change in pull request #5014: NIFI-8442 Convert Timestamp, Date and Time to String

[GitBox <gi...@apache.org>]

pvillard31 commented on a change in pull request #5014:
URL: https://github.com/apache/nifi/pull/5014#discussion_r671573920



##########
File path: nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery/PutBigQueryStreaming.java
##########
@@ -18,10 +18,11 @@
 package org.apache.nifi.processors.gcp.bigquery;
 
 import java.io.InputStream;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
+import java.sql.Date;
+import java.sql.Time;
+import java.sql.Timestamp;
+import java.text.SimpleDateFormat;
+import java.util.*;

Review comment:
       ```suggestion
   import java.sql.Date;
   import java.sql.Time;
   import java.sql.Timestamp;
   import java.text.SimpleDateFormat;
   import java.util.ArrayList;
   import java.util.HashMap;
   import java.util.List;
   import java.util.Map;
   import java.util.TimeZone;
   ```

##########
File path: nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/bigquery/PutBigQueryStreaming.java
##########
@@ -191,6 +192,18 @@ public void onTrigger(ProcessContext context, ProcessSession session) throws Pro
                     lmapr.add(convertMapRecord(((MapRecord) mapr).toMap()));
                 }
                 result.put(key, lmapr);
+            } else if (obj instanceof Timestamp) {
+                SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+                format.setTimeZone(TimeZone.getTimeZone("UTC"));
+                result.put(key, format.format(((Timestamp) obj).getTime()));
+            } else if (obj instanceof Time) {
+                SimpleDateFormat format = new SimpleDateFormat("HH:mm:ss");
+                format.setTimeZone(TimeZone.getTimeZone("UTC"));
+                result.put(key, format.format(((Time) obj).getTime()));
+            } else if (obj instanceof Date) {
+                SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd");
+                format.setTimeZone(TimeZone.getTimeZone("UTC"));
+                result.put(key, format.format(((Date) obj).getTime()));

Review comment:
       When executing the integration test you added, I see the below data being added in BQ:
   
   <img width="858" alt="Screenshot 2021-07-17 at 02 01 35" src="https://user-images.githubusercontent.com/11541012/126018935-4da2b128-6c1a-4538-a5a2-4dd1a8e6a490.png">
   
   Which makes the test fail because:
   ````java
   assertEquals(john.get("birth").getRecordValue().get(0).getStringValue(), "1995-06-01");
   assertEquals(john.get("birth").getRecordValue().get(1).getStringValue(), "15:57:27");
   assertEquals(john.get("birth").getRecordValue().get(2).getTimestampValue() / 1000, 802022247000L);
   ````
   
   Something needs to be fixed.

##########
File path: nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/test/java/org/apache/nifi/processors/gcp/bigquery/PutBigQueryStreamingIT.java
##########
@@ -21,12 +21,15 @@
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertTrue;
 
+import java.nio.file.Files;
 import java.nio.file.Paths;
 import java.util.Iterator;
 
+import com.google.api.client.util.DateTime;

Review comment:
       ```suggestion
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org