You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gobblin.apache.org by su...@apache.org on 2020/03/12 17:29:06 UTC
[incubator-gobblin] branch master updated: [GOBBLIN-1081] Adding
support of timestamp data type for CsvToJsonConverter
This is an automated email from the ASF dual-hosted git repository.
suvasude pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-gobblin.git
The following commit(s) were added to refs/heads/master by this push:
new 120a109 [GOBBLIN-1081] Adding support of timestamp data type for CsvToJsonConverter
120a109 is described below
commit 120a1097dcc4acbf2d72ed0b15c48c04f09a5a0c
Author: Haoji Liu <ha...@haojliu-mn1.linkedin.biz>
AuthorDate: Thu Mar 12 10:29:00 2020 -0700
[GOBBLIN-1081] Adding support of timestamp data type for CsvToJsonConverter
Currently CsvToJsonConverterV2 only supprot
string, bool, double and int.
Need to support Avro logic type timestamp
Test Done:
1. unit test
2. integration test by running a Gobblin job with
converter CsvToJsonConverterV2, and set one column
to timestamp
Closes #2920 from haojiliu/gaas
---
.../java/org/apache/gobblin/converter/csv/CsvToJsonConverterV2.java | 6 ++++++
.../src/test/resources/converter/csv/schema_with_10_fields.json | 2 +-
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/gobblin-core/src/main/java/org/apache/gobblin/converter/csv/CsvToJsonConverterV2.java b/gobblin-core/src/main/java/org/apache/gobblin/converter/csv/CsvToJsonConverterV2.java
index e69f277..59c8973 100644
--- a/gobblin-core/src/main/java/org/apache/gobblin/converter/csv/CsvToJsonConverterV2.java
+++ b/gobblin-core/src/main/java/org/apache/gobblin/converter/csv/CsvToJsonConverterV2.java
@@ -265,6 +265,12 @@ public class CsvToJsonConverterV2 extends Converter<String, JsonArray, String[],
JsonElement convert(String value) {
return new JsonPrimitive(value);
}
+ },
+ TIMESTAMP {
+ @Override
+ JsonElement convert(String value) {
+ return new JsonPrimitive(value);
+ }
};
abstract JsonElement convert(String value);
}
diff --git a/gobblin-core/src/test/resources/converter/csv/schema_with_10_fields.json b/gobblin-core/src/test/resources/converter/csv/schema_with_10_fields.json
index fe4b55a..bcf94ea 100644
--- a/gobblin-core/src/test/resources/converter/csv/schema_with_10_fields.json
+++ b/gobblin-core/src/test/resources/converter/csv/schema_with_10_fields.json
@@ -4,7 +4,7 @@
"comment": "",
"isNullable": "true",
"dataType": {
- "type": "string"
+ "type": "timestamp"
}
},
{