You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2011/08/16 07:56:27 UTC
[jira] [Created] (HIVE-2380) Add ByteArray Datatype
Add ByteArray Datatype
----------------------
Key: HIVE-2380
URL: https://issues.apache.org/jira/browse/HIVE-2380
Project: Hive
Issue Type: New Feature
Components: Serializers/Deserializers
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: hive-2380_2.patch
Same patch as on RB.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118232#comment-13118232 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
@John
bq. I would be a lot more comfortable with this patch if it included testing of real binary data (not just strings encoded as binary) at the QL level end-to-end, getting stored in a table, queried, passed through UDF's, etc. (not just the Java unit test level of serdes) since that is the intended use case
Added following tests:
* ba_table1.q basic test which tests a) binary type works correctly in grammar b) string can be cast into binary c) binary can be stored in a table d) binary data can be loaded back again and queried d) order-by on a binary key
* ba_table2.q All the test in ba_test1.q + using LazyBinarySerde instead of LazySimpleSerde
* ba_table3.q All the tests of ba_table1.q + test for a group-by and aggregation on a binary key.
* ba_table_udfs.q this query tests all the udfs provided to work with binary works.
* ba_table_union.q this query tests ba_table1.q + nested queries with multiple operations on binary data types + union on binary types
* binary_table_colserde.q - Everything in ba_table1.q + columnar serde in RCFILE.
* binary_table_bincolserde.q Tests everything in binary_table_colserde.q + uses LazyBinaryColumnarSerde
* load_binary_data.q this query loads native binary data, stores in a table and then queries it. Note that string.txt contains binary data. Also uses transform clause and then length udf on binary fields.
bq. need negative tests for attempting illegal conversions to/from binary
Added whole bunch of negative tests of form invalid_cast_to_binary_[1..6].q and invalid_cast_from_binary_[1..6].q to tests illegal conversion attempts.
bq. please rename this issue since we're going with BINARY instead of BYTEARRAY
Done.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Status: Patch Available (was: Open)
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107357#comment-13107357 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/
-----------------------------------------------------------
(Updated 2011-09-18 05:52:41.197110)
Review request for hive.
Changes
-------
Updated patch incorporating feedback
Summary
-------
This patch adds bytearray as a new datatype in Hive.
This addresses bug HIVE-2380.
https://issues.apache.org/jira/browse/HIVE-2380
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1172168
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
trunk/serde/if/serde.thrift 1172168
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1172168
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1172168
trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1172168
trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1172168
trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1172168
trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1172168
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1172168
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1172168
Diff: https://reviews.apache.org/r/1586/diff
Testing
-------
Added and updated unit tests.
Added new system tests.
Thanks,
Ashutosh
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118223#comment-13118223 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/
-----------------------------------------------------------
(Updated 2011-09-30 17:37:25.293739)
Review request for hive.
Changes
-------
Incorporating John's feedback + lot of tests.
Summary
-------
This patch adds bytearray as a new datatype in Hive.
This addresses bug HIVE-2380.
https://issues.apache.org/jira/browse/HIVE-2380
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_6.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_6.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table3.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_union.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/binary_table_bincolserde.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/binary_table_colserde.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/load_binary_data.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_1.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_2.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_3.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_4.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_5.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_6.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table3.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_union.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/binary_table_bincolserde.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/binary_table_colserde.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/load_binary_data.q.out PRE-CREATION
trunk/serde/if/serde.thrift 1177528
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1177528
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1177528
trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1177528
trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1177528
trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1177528
trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/DelimitedJSONSerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1177528
Diff: https://reviews.apache.org/r/1586/diff
Testing
-------
Added and updated unit tests.
Added new system tests.
Thanks,
Ashutosh
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104950#comment-13104950 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
Turned out binary is already reserved KW in hive grammar. But, looks like its not used anywhere? Can someone confirm. If so, I will use that KW, else we need to find a different name.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2380) Add Binary Datatype in Hive
Posted by "John Sichi (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi resolved HIVE-2380.
------------------------------
Resolution: Fixed
Fix Version/s: 0.9.0
Release Note: New datatype BINARY can now be used in column definitions.
Hadoop Flags: Reviewed
Committed to trunk. Thanks Ashutosh!
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.9.0
>
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: hive-2380_4.patch
Sorry about that. Those test cases needs to be updated to match the updated description of those udfs. Attaching latest patch which includes updated *.q.out files for those udfs.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098082#comment-13098082 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
bq. Is there a design doc somewhere?
Not yet. I can put together some design notes quickly over here. Is there an example of design docs for the features done previously for Hive which I can look at to get an idea what shall I cover in it?
bq. Since Hive already has an array type, but this feature is independent, we probably want a different type name than bytearray.
OK. Binary ?
bq. For conversions, is going through string for all types a good default behavior? An alternative would be to prevent implicit conversions altogether, and force users to pick the UDF with the desired behavior. E.g. for string/binary conversion, it's a good idea to be able to specify an encoding rather than always using the JVM default.
I also thought about casting and wasn't inclined for adding implicit casting. But I went with it so as to make easy things easier. This way users can use this type easily in scripts without needing to insert casts every time and in cases this doesnt work they can always write udfs. Further, in many cases, JVM encoding is a good default. But, if you think thats not a good idea, I can take away implicit casting.
bq. How does the new type work with
bq. TRANSFORM scripts,
I am assuming providing toString() will be good enough to make sure we can send data in string form and after receiving can convert into bytearray. Is there anything else ?
bq. UDF's,
Like other types. Do I need to think about anything here?
bq. saving to textfile, etc?
I assume you mean file containing text data in other columns, if user does so, it will be his responsibility to escape and format data appropriately so that he can load it later, potentially with the serde which does understand the format and escaping.
bq. Don't we need more accessor functions (e.g. making the existing string functions such as LENGTH work)?
Length should be possible. Any other accessor functions?
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118224#comment-13118224 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java, line 675
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41673#file41673line675>
bq. >
bq. > I thought we decided an explicit cast would be required even for string<-->binary?
Correct. Accidentally left. Will remove it in the new patch.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java, line 36
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41678#file41678line36>
bq. >
bq. > @Description annotation for this class needs to be updated.
Will update in new patch.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java, line 55
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41679#file41679line55>
bq. >
bq. > @Description annotation for this class needs to be updated.
Will update in new patch.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java, line 29
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41682#file41682line29>
bq. >
bq. > This UDF class needs an @Description annotation.
Will update in new patch.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java, line 43
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41680#file41680line43>
bq. >
bq. > @Description annotation for this class needs to be updated.
Will update in new patch.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java, line 67
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41682#file41682line67>
bq. >
bq. > The CAST should have parentheses, so
bq. >
bq. > CAST(x AS BINARY)
bq. >
bq. > (this applies to the existing code in GenericUDFTimestamp too)
bq. >
Will update in new patch.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/ql/src/test/queries/clientpositive/ba_table1.q, line 9
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41683#file41683line9>
bq. >
bq. > Use ORDER BY on a key for all queries to guarantee test determinism. Also, since test output keeps getting bigger and bigger, it's best to filter down to just a few rows to keep it easier to review and manage.
bq. >
bq. > (This comment applies to all tests in this patch.)
Added order-by to guarantee determinism and limit to keep number of rows small in all the tests.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java, line 267
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41696#file41696line267>
bq. >
bq. > Is there a test for this somewhere? It seems like a case where we need base64.
Missed the changes in DelimitedJSONserde so it was not getting used. DelimitedJSONserde now uses this method to print, so its gets used by all the tests.
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java, line 83
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41698#file41698line83>
bq. >
bq. > I'm not sure about the original connection between UTF and strings here (was something forcing a UTF encoding), but is it valid for arbitrary binary data?
You are correct, this may break when there is a binary data of exactly one byte long and with that bit pattern, it will return null instead of that bit pattern. But thats a pretty corner-case. We can document something along the line "If there is a possibility that your binary data may contain exactly one byte of 10111111, don't use LazyBinaryColumnarSerde with binary data type."
bq. On 2011-09-27 00:43:03, John Sichi wrote:
bq. > trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java, line 393
bq. > <https://reviews.apache.org/r/1586/diff/2/?file=41716#file41716line393>
bq. >
bq. > Since we decided we're not going to support implicit conversions, these don't belong here, right?
Correct. Removed.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/#review2076
-----------------------------------------------------------
On 2011-09-18 05:52:41, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1586/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-18 05:52:41)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch adds bytearray as a new datatype in Hive.
bq.
bq.
bq. This addresses bug HIVE-2380.
bq. https://issues.apache.org/jira/browse/HIVE-2380
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
bq. trunk/serde/if/serde.thrift 1172168
bq. trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1172168
bq. trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1172168
bq. trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1172168
bq. trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1172168
bq. trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1172168
bq. trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1172168
bq.
bq. Diff: https://reviews.apache.org/r/1586/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added and updated unit tests.
bq. Added new system tests.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104844#comment-13104844 ]
John Sichi commented on HIVE-2380:
----------------------------------
Just checked, and the name BINARY in JDBC and standard SQL corresponds to a fixed-length binary type with a specified precision:
http://download.oracle.com/javase/1.3/docs/guide/jdbc/getstart/mapping.html
But I'm fine with us using it to mean something slightly different, and matching thrift.
Regarding the conversions, it's fine to break it out into multiple patches; open follow-up JIRA's linked to this one. I think the hex one will be very useful for debugging.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115081#comment-13115081 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/#review2076
-----------------------------------------------------------
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
<https://reviews.apache.org/r/1586/#comment4695>
I thought we decided an explicit cast would be required even for string<-->binary?
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java
<https://reviews.apache.org/r/1586/#comment4699>
@Description annotation for this class needs to be updated.
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java
<https://reviews.apache.org/r/1586/#comment4698>
@Description annotation for this class needs to be updated.
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java
<https://reviews.apache.org/r/1586/#comment4700>
@Description annotation for this class needs to be updated.
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java
<https://reviews.apache.org/r/1586/#comment4697>
This UDF class needs an @Description annotation.
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java
<https://reviews.apache.org/r/1586/#comment4696>
The CAST should have parentheses, so
CAST(x AS BINARY)
(this applies to the existing code in GenericUDFTimestamp too)
trunk/ql/src/test/queries/clientpositive/ba_table1.q
<https://reviews.apache.org/r/1586/#comment4701>
Use ORDER BY on a key for all queries to guarantee test determinism. Also, since test output keeps getting bigger and bigger, it's best to filter down to just a few rows to keep it easier to review and manage.
(This comment applies to all tests in this patch.)
trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java
<https://reviews.apache.org/r/1586/#comment4708>
Is there a test for this somewhere? It seems like a case where we need base64.
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java
<https://reviews.apache.org/r/1586/#comment4715>
I'm not sure about the original connection between UTF and strings here (was something forcing a UTF encoding), but is it valid for arbitrary binary data?
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
<https://reviews.apache.org/r/1586/#comment4716>
Since we decided we're not going to support implicit conversions, these don't belong here, right?
- John
On 2011-09-18 05:52:41, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1586/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-18 05:52:41)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch adds bytearray as a new datatype in Hive.
bq.
bq.
bq. This addresses bug HIVE-2380.
bq. https://issues.apache.org/jira/browse/HIVE-2380
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1172168
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
bq. trunk/serde/if/serde.thrift 1172168
bq. trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1172168
bq. trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1172168
bq. trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1172168
bq. trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1172168
bq. trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1172168
bq. trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1172168
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1172168
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1172168
bq.
bq. Diff: https://reviews.apache.org/r/1586/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added and updated unit tests.
bq. Added new system tests.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Chuck Connell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507761#comment-13507761 ]
Chuck Connell commented on HIVE-2380:
-------------------------------------
I am trying to use this feature (BINARY columns) and I believe I have the perfect use-case for it, but I am missing something.
Here is the background... I have some files that each contain just one logical field, which is a binary object. (The files are Google Protobuf format.) I want to put these binary files into a larger file, where each protobuf is a logical record. Then I want to define a Hive table that stores each protobuf as one row, with the entire protobuf object in one BINARY column. Then I will use a custom UDF to select/query the binary object.
This is about as simple as can be for putting binary data into Hive. But all of the test cases for this jira seem to draw the binary columns from another existing table and CAST them. I want to load the files from disk.
What file format should I use to package the binary rows? What should the Hive table definition be? I cannot use TEXTFILE, since the binary may contain newlines. Many of my attempts have choked on the newlines.
Thanks very much,
Chuck Connell
Nuance
Burlington, MA
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch, hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Status: Patch Available (was: Open)
Marking as Patch Available as its ready for review.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098322#comment-13098322 ]
John Sichi commented on HIVE-2380:
----------------------------------
You can find design doc examples here:
https://cwiki.apache.org/confluence/display/Hive/DesignDocs
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104815#comment-13104815 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
@John,
For conversions, in this jira I am planning to provide functionality for to/from string using default JVM encoding, rest of these conversions I will take up in a follow-up jira. Motivation is to avoid having one giant patch and have multiple smaller patches. Sounds reasonable?
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Hojjat Jafarpour (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189346#comment-13189346 ]
Hojjat Jafarpour commented on HIVE-2380:
----------------------------------------
Is there an example for defining a table with binary and timestamp columns and loading daata into the table and querying it?
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Summary: Add Binary Datatype in Hive (was: Add ByteArray Datatype)
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103243#comment-13103243 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
Initial design proposal is here:
https://cwiki.apache.org/confluence/display/Hive/Binary+DataType+Proposal
Comments welcome.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: hive-2380_1.patch
Updated patch incorporating feedback.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119835#comment-13119835 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/
-----------------------------------------------------------
(Updated 2011-10-04 01:10:59.091568)
Review request for hive.
Summary
-------
This patch adds bytearray as a new datatype in Hive.
This addresses bug HIVE-2380.
https://issues.apache.org/jira/browse/HIVE-2380
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java 1177528
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_6.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_6.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table3.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_union.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/binary_table_bincolserde.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/binary_table_colserde.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/load_binary_data.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_1.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_2.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_3.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_4.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_5.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_6.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table3.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_union.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/binary_table_bincolserde.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/binary_table_colserde.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/load_binary_data.q.out PRE-CREATION
trunk/serde/if/serde.thrift 1177528
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1177528
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1177528
trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1177528
trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1177528
trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1177528
trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/DelimitedJSONSerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1177528
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1177528
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1177528
Diff: https://reviews.apache.org/r/1586/diff
Testing
-------
Added and updated unit tests.
Added new system tests.
Thanks,
Ashutosh
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add Binary Datatype in Hive
Posted by "John Sichi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2380:
-----------------------------
Status: Open (was: Patch Available)
ant test hit the following failures:
TestCliDriver:
show_describe_func_quotes
show_functions
udf_concat
udf_length
udf_substr
udf_substring
TestParseNegative:
invalid_function_param2
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104085#comment-13104085 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
Is name binary acceptable?
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2380:
-----------------------------
Status: Open (was: Patch Available)
Some comments added on Review Board. Also:
* I would be a lot more comfortable with this patch if it included testing of real binary data (not just strings encoded as binary) at the QL level end-to-end, getting stored in a table, queried, passed through UDF's, etc. (not just the Java unit test level of serdes) since that is the intended use case
* could you open followup JIRA issues for any items which you intend to address as immediate followups to this patch?
* need negative tests for attempting illegal conversions to/from binary
* please rename this issue since we're going with BINARY instead of BYTEARRAY
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: hive-2380.patch
Complete patch. It is ready for review.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120588#comment-13120588 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
@John created and linked new jiras for follow-up work.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2380:
-----------------------------
Status: Open (was: Patch Available)
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103894#comment-13103894 ]
John Sichi commented on HIVE-2380:
----------------------------------
For accessor functions:
* length
* substring
* concat
We can follow up later with search capabilities.
For conversions:
* to/from hex string
* to/from string using a specific encoding (or default JVM encoding if not specified)
* to/from base64 string
We can follow up later with more interesting conversions for non-string types.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094245#comment-13094245 ]
John Sichi commented on HIVE-2380:
----------------------------------
Taking a look.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119838#comment-13119838 ]
Ashutosh Chauhan commented on HIVE-2380:
----------------------------------------
Updated RB with latest patch.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095669#comment-13095669 ]
John Sichi commented on HIVE-2380:
----------------------------------
Ashutosh, maybe we can discuss this one at the contributor meetup next week (and then record the conclusions here).
A few questions that I've heard so far:
* Is there a design doc somewhere?
* Since Hive already has an array type, but this feature is independent, we probably want a different type name than bytearray.
* For conversions, is going through string for all types a good default behavior? An alternative would be to prevent implicit conversions altogether, and force users to pick the UDF with the desired behavior. E.g. for string/binary conversion, it's a good idea to be able to specify an encoding rather than always using the JVM default.
* How does the new type work with TRANSFORM scripts, UDF's, saving to textfile, etc?
* Don't we need more accessor functions (e.g. making the existing string functions such as LENGTH work)?
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122249#comment-13122249 ]
Hudson commented on HIVE-2380:
------------------------------
Integrated in Hive-trunk-h0.21 #997 (See [https://builds.apache.org/job/Hive-trunk-h0.21/997/])
HIVE-2380. Add Binary Datatype in Hive
(Ashutosh Chauhan via jvs)
jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179784
Files :
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_1.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_2.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_3.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_4.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_5.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_6.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_1.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_2.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_3.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_4.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_5.q
* /hive/trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_6.q
* /hive/trunk/ql/src/test/queries/clientpositive/ba_table1.q
* /hive/trunk/ql/src/test/queries/clientpositive/ba_table2.q
* /hive/trunk/ql/src/test/queries/clientpositive/ba_table3.q
* /hive/trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q
* /hive/trunk/ql/src/test/queries/clientpositive/ba_table_union.q
* /hive/trunk/ql/src/test/queries/clientpositive/binary_table_bincolserde.q
* /hive/trunk/ql/src/test/queries/clientpositive/binary_table_colserde.q
* /hive/trunk/ql/src/test/queries/clientpositive/load_binary_data.q
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_1.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_2.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_3.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_4.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_5.q.out
* /hive/trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ba_table1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ba_table2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ba_table3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ba_table_union.q.out
* /hive/trunk/ql/src/test/results/clientpositive/binary_table_bincolserde.q.out
* /hive/trunk/ql/src/test/results/clientpositive/binary_table_colserde.q.out
* /hive/trunk/ql/src/test/results/clientpositive/load_binary_data.q.out
* /hive/trunk/ql/src/test/results/clientpositive/show_describe_func_quotes.q.out
* /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_concat.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_length.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_substr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_substring.q.out
* /hive/trunk/ql/src/test/results/compiler/errors/invalid_function_param2.q.out
* /hive/trunk/serde/if/serde.thrift
* /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp
* /hive/trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h
* /hive/trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java
* /hive/trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php
* /hive/trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py
* /hive/trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/DelimitedJSONSerDe.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java
* /hive/trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.9.0
>
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: (was: hive-2380.patch)
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087435#comment-13087435 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/
-----------------------------------------------------------
Review request for hive.
Summary
-------
This patch adds bytearray as a new datatype in Hive.
This addresses bug HIVE-2380.
https://issues.apache.org/jira/browse/HIVE-2380
Diffs
-----
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableByteArrayObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableByteArrayObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableVoidObjectInspector.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1159452
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/ByteArrayObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaByteArrayObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryByteArray.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyByteArrayObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyByteArray.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToByte.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToDouble.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToFloat.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToInteger.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToLong.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToShort.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/UDFToByteArray.java PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
trunk/serde/if/serde.thrift 1159452
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1159452
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1159452
trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1159452
trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1159452
trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1159452
trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1159452
trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1159452
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1159452
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1159452
Diff: https://reviews.apache.org/r/1586/diff
Testing
-------
Added and updated unit tests.
Added new system tests.
Thanks,
Ashutosh
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Status: Patch Available (was: Open)
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121383#comment-13121383 ]
John Sichi commented on HIVE-2380:
----------------------------------
I'll rerun with latest. Please remember to click "Submit Patch" to get it back into the queue.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121301#comment-13121301 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/
-----------------------------------------------------------
(Updated 2011-10-05 18:05:13.442838)
Review request for hive.
Changes
-------
Uploading hive-2380_4.patch which includes updated *.q.out files for udfs whose description got updated.
Summary
-------
This patch adds bytearray as a new datatype in Hive.
This addresses bug HIVE-2380.
https://issues.apache.org/jira/browse/HIVE-2380
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java 1179305
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_6.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_1.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_2.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_3.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_4.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_5.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_6.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table3.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/ba_table_union.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/binary_table_bincolserde.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/binary_table_colserde.q PRE-CREATION
trunk/ql/src/test/queries/clientpositive/load_binary_data.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_1.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_2.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_3.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_4.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_5.q.out PRE-CREATION
trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_6.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table3.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/ba_table_union.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/binary_table_bincolserde.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/binary_table_colserde.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/load_binary_data.q.out PRE-CREATION
trunk/ql/src/test/results/clientpositive/show_describe_func_quotes.q.out 1179305
trunk/ql/src/test/results/clientpositive/show_functions.q.out 1179305
trunk/ql/src/test/results/clientpositive/udf_concat.q.out 1179305
trunk/ql/src/test/results/clientpositive/udf_length.q.out 1179305
trunk/ql/src/test/results/clientpositive/udf_substr.q.out 1179305
trunk/ql/src/test/results/clientpositive/udf_substring.q.out 1179305
trunk/ql/src/test/results/compiler/errors/invalid_function_param2.q.out 1179305
trunk/serde/if/serde.thrift 1179305
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1179305
trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1179305
trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1179305
trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1179305
trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1179305
trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/DelimitedJSONSerDe.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1179305
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1179305
trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1179305
Diff: https://reviews.apache.org/r/1586/diff
Testing
-------
Added and updated unit tests.
Added new system tests.
Thanks,
Ashutosh
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113770#comment-13113770 ]
John Sichi commented on HIVE-2380:
----------------------------------
I'm planning to review this one next week.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118527#comment-13118527 ]
jiraposter@reviews.apache.org commented on HIVE-2380:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1586/#review2239
-----------------------------------------------------------
Looking good (added a note for one missing ORDER BY remaining).
But regarding "10111111", I don't think we should have any corner cases at all in the set of binary strings which can be stored. That would just be a land mine. Can you think of a solution?
trunk/ql/src/test/queries/clientpositive/ba_table_union.q
<https://reviews.apache.org/r/1586/#comment5195>
Need ORDER BY.
- John
On 2011-09-30 17:37:25, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1586/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-30 17:37:25)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch adds bytearray as a new datatype in Hive.
bq.
bq.
bq. This addresses bug HIVE-2380.
bq. https://issues.apache.org/jira/browse/HIVE-2380
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFConcat.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLength.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFSubstr.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTimestamp.java 1177528
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToBinary.java PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_1.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_2.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_3.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_4.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_5.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_from_binary_6.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_1.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_2.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_3.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_4.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_5.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientnegative/invalid_cast_to_binary_6.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table1.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table2.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table3.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/ba_table_union.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/binary_table_bincolserde.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/binary_table_colserde.q PRE-CREATION
bq. trunk/ql/src/test/queries/clientpositive/load_binary_data.q PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_1.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_2.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_3.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_4.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_5.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_from_binary_6.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_1.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_2.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_3.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_4.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_5.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientnegative/invalid_cast_to_binary_6.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table1.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table2.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table3.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/ba_table_union.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/binary_table_bincolserde.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/binary_table_colserde.q.out PRE-CREATION
bq. trunk/ql/src/test/results/clientpositive/load_binary_data.q.out PRE-CREATION
bq. trunk/serde/if/serde.thrift 1177528
bq. trunk/serde/src/gen/thrift/gen-cpp/serde_constants.h 1177528
bq. trunk/serde/src/gen/thrift/gen-cpp/serde_constants.cpp 1177528
bq. trunk/serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java 1177528
bq. trunk/serde/src/gen/thrift/gen-php/serde/serde_constants.php 1177528
bq. trunk/serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 1177528
bq. trunk/serde/src/gen/thrift/gen-rb/serde_constants.rb 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/DelimitedJSONSerDe.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyBinary.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryBinary.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/BinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 1177528
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableBinaryObjectInspector.java PRE-CREATION
bq. trunk/serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/TestStatsSerde.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/MyTestClass.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/binarysortable/TestBinarySortableSerDe.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/columnar/TestLazyBinaryColumnarSerDe.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazyPrimitive.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazy/TestLazySimpleSerDe.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/MyTestClassBigger.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorConverters.java 1177528
bq. trunk/serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestStandardObjectInspectors.java 1177528
bq.
bq. Diff: https://reviews.apache.org/r/1586/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added and updated unit tests.
bq. Added new system tests.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120476#comment-13120476 ]
John Sichi commented on HIVE-2380:
----------------------------------
+1. Will commit when tests pass.
Ashutosh, could you please open and link these followups:
* UDF's to convert to/from hex string
* UDF's to convert to/from string using a specific encoding
* UDF's to convert to/from base64 string
* UDF's to convert to/from non-string types using a particular serde
* make LazyBinaryColumnarSerDe support 0-length binary strings, assigned to Krishna Kumar (see HIVE-956) in case he wants to work on a way to remove the limitation
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add ByteArray Datatype
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: hive-2380.patch
This is WIP. Attaching it early to illustrate the design to get feedback.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119827#comment-13119827 ]
John Sichi commented on HIVE-2380:
----------------------------------
Can you update review board with latest?
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2380:
-----------------------------------
Attachment: hive-2380_3.patch
Attaching new patch which addresses John's two concerns. Added order-by in union query and for
bq. But regarding "10111111", I don't think we should have any corner cases at all in the set of binary strings which can be stored. That would just be a land mine. Can you think of a solution?
Instead of special casing for the case of serializing zero length binary throwing an exception. With that, if there ever is a case when user tries to serialize a 0-length binary through BinaryColumnarSerde, we will throw an exception. Now, there is no need of special casing in LazyBinaryColumnarSerde and LazyBinaryColumnarSerde, so rolled back changes in those. Also, added a unit test case testLazyBinaryColumnarSerdeWithEmpyBinary()to exercise this code path.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch, hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add ByteArray Datatype
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104980#comment-13104980 ]
John Sichi commented on HIVE-2380:
----------------------------------
I don't see any references to it, so I think you're free to use it.
> Add ByteArray Datatype
> ----------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2380) Add Binary Datatype in Hive
Posted by "Chuck Connell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507810#comment-13507810 ]
Chuck Connell commented on HIVE-2380:
-------------------------------------
Let me state my earlier comment another way... If the BINARY column type "works" in narrow sense, but it is impossible to get binary data from a file into Hive, then this feature is essentially useless. So we need some explanation for how to use the existing DDL features to support BINARY columns, or a new file format that allows it.
> Add Binary Datatype in Hive
> ---------------------------
>
> Key: HIVE-2380
> URL: https://issues.apache.org/jira/browse/HIVE-2380
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: hive-2380_1.patch, hive-2380_2.patch, hive-2380_3.patch, hive-2380_4.patch, hive-2380.patch
>
>
> Add bytearray as a primitive data type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira