You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Zoltan Haindrich <ki...@rxd.hu> on 2018/06/15 14:13:00 UTC
Re: Review Request 65480: HIVE-18545 Add UDF to parse complex types
from json
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65480/
-----------------------------------------------------------
(Updated June 15, 2018, 2:12 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
use the same in serde/udf; some refactor
Bugs: HIVE-18545
https://issues.apache.org/jira/browse/HIVE-18545
Repository: hive-git
Description (updated)
-------
* add a generalized reader for json2hive struct parsing
* add json_read udf ; tests
* rename the new "copied" the new jsonserde "serde2.JsonSerde" to "serde2.json.JsonSerde2" - I think it would be bad to have 2 classes named the same ; in the same project, it would even confuse users - because only the package path was different
* make the hcat jsonserde use the "serde2" version as a back end - so that only 1 real jsonserde exists in hive right now...
Diffs (updated)
-----
hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/JsonSerDe.java 114c205c83
hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestJsonSerDe.java 8aeb4f4f10
itests/hive-jmh/src/main/java/org/apache/hive/benchmark/udf/json_read/JsonReadBench.java PRE-CREATION
itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.json PRE-CREATION
itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.type PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e77fe18129
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFJsonRead.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFJsonRead.java PRE-CREATION
ql/src/test/queries/clientpositive/json_serde2.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_json_read.q PRE-CREATION
ql/src/test/results/clientpositive/json_serde2.q.out PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out 91d3660aa0
ql/src/test/results/clientpositive/udf_json_read.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/JsonSerDe.java 1119fa2af5
serde/src/java/org/apache/hadoop/hive/serde2/json/HiveJsonStructReader.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/json/JsonSerDe2.java PRE-CREATION
streaming/src/java/org/apache/hive/streaming/StrictJsonWriter.java cabb64cdcc
Diff: https://reviews.apache.org/r/65480/diff/2/
Changes: https://reviews.apache.org/r/65480/diff/1-2/
Testing (updated)
-------
* added some new tests for the udf
Thanks,
Zoltan Haindrich
Re: Review Request 65480: HIVE-18545 Add UDF to parse complex types
from json
Posted by Zoltan Haindrich <ki...@rxd.hu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65480/
-----------------------------------------------------------
(Updated June 19, 2018, 7:02 a.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
update patch6; does not moves serde2.jsonserde
Bugs: HIVE-18545
https://issues.apache.org/jira/browse/HIVE-18545
Repository: hive-git
Description (updated)
-------
* add a generalized reader for json2hive struct parsing
* add json_read udf ; tests
* make the hcat jsonserde use the "serde2" version as a back end - so that only 1 real jsonserde exists in hive right now...
Diffs (updated)
-----
hcatalog/core/src/main/java/org/apache/hive/hcatalog/data/JsonSerDe.java 114c205c83
hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestJsonSerDe.java 8aeb4f4f10
itests/hive-jmh/src/main/java/org/apache/hive/benchmark/udf/json_read/JsonReadBench.java PRE-CREATION
itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.json PRE-CREATION
itests/hive-jmh/src/main/resources/org/apache/hive/benchmark/udf/json_read/val1.type PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e77fe18129
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFJsonRead.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFJsonRead.java PRE-CREATION
ql/src/test/queries/clientpositive/json_serde2.q PRE-CREATION
ql/src/test/queries/clientpositive/udf_json_read.q PRE-CREATION
ql/src/test/results/clientpositive/json_serde2.q.out PRE-CREATION
ql/src/test/results/clientpositive/show_functions.q.out 91d3660aa0
ql/src/test/results/clientpositive/udf_json_read.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/JsonSerDe.java 1119fa2af5
serde/src/java/org/apache/hadoop/hive/serde2/json/HiveJsonStructReader.java PRE-CREATION
streaming/src/java/org/apache/hive/streaming/StrictJsonWriter.java cabb64cdcc
Diff: https://reviews.apache.org/r/65480/diff/3/
Changes: https://reviews.apache.org/r/65480/diff/2-3/
Testing
-------
* added some new tests for the udf
Thanks,
Zoltan Haindrich