You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Mohammad Islam <mi...@yahoo.com> on 2013/12/17 01:00:37 UTC
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/
-----------------------------------------------------------
(Updated Dec. 17, 2013, midnight)
Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
Changes
-------
Fix the failed test case.
Bugs: HIVE-5829
https://issues.apache.org/jira/browse/HIVE-5829
Repository: hive-git
Description
-------
Rewrite the UDFS *pads and *trim using GenericUDF.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLTrim.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLpad.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFRTrim.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFRpad.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFTrim.java PRE-CREATION
Diff: https://reviews.apache.org/r/15654/diff/
Testing
-------
Thanks,
Mohammad Islam
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On Dec. 18, 2013, 5:37 a.m., Xuefu Zhang wrote:
> > ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java, line 774
> > <https://reviews.apache.org/r/15654/diff/4/?file=399245#file399245line774>
> >
> > I don't think we need the bridge udf for generic UDFs.
This change is only to replace UDFLTrim with new GenericUDFLTrim used in this test case. Generic bridge UDF is already there.
> On Dec. 18, 2013, 5:37 a.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java, line 109
> > <https://reviews.apache.org/r/15654/diff/4/?file=399238#file399238line109>
> >
> > I'm not sure if this is intentional, but the logic here means that any of the three input can have a type of INT. If INT is okay, then why not BYTE, SHORT, or LONG? It's probably better to check each argument's type separately.
will do.
> On Dec. 18, 2013, 5:37 a.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java, line 48
> > <https://reviews.apache.org/r/15654/diff/4/?file=399238#file399238line48>
> >
> > Msg doesn't match the if condition.
will correct.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30604
-----------------------------------------------------------
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30604
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java
<https://reviews.apache.org/r/15654/#comment58615>
Msg doesn't match the if condition.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java
<https://reviews.apache.org/r/15654/#comment58616>
I'm not sure if this is intentional, but the logic here means that any of the three input can have a type of INT. If INT is okay, then why not BYTE, SHORT, or LONG? It's probably better to check each argument's type separately.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java
<https://reviews.apache.org/r/15654/#comment58617>
instanceof might be better used here as you would need exception handling.
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java
<https://reviews.apache.org/r/15654/#comment58618>
I don't think we need the bridge udf for generic UDFs.
- Xuefu Zhang
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30726
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java
<https://reviews.apache.org/r/15654/#comment58824>
I was thinking of doing this:
if (!(auguments[0] instanceof PrimitiveObjectInspector)) {
...
}
- Xuefu Zhang
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30669
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java
<https://reviews.apache.org/r/15654/#comment58764>
Would you please further explain on this? preferably with an example.
- Mohammad Islam
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On Dec. 18, 2013, 10:58 a.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java, line 78
> > <https://reviews.apache.org/r/15654/diff/4/?file=399238#file399238line78>
> >
> > Having gone through some pain with Hive on Windows, the bytes returned by String.getBytes() will not be in utf-8 if the default encoding is something other than utf-8. Would be safer here to either use getBytes("UTF-8"), or Text.encode() if you want to get bytes from the string. Or just do the padding as Strings.
"str" is of type "Text". It doesn't have getBytes("UTF-8"). only have getBytes().
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30610
-----------------------------------------------------------
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30610
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java
<https://reviews.apache.org/r/15654/#comment58620>
Having gone through some pain with Hive on Windows, the bytes returned by String.getBytes() will not be in utf-8 if the default encoding is something other than utf-8. Would be safer here to either use getBytes("UTF-8"), or Text.encode() if you want to get bytes from the string. Or just do the padding as Strings.
- Jason Dere
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Xuefu Zhang <xz...@cloudera.com>.
> On Dec. 20, 2013, 2:37 a.m., Mohammad Islam wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java, line 47
> > <https://reviews.apache.org/r/15654/diff/4/?file=399239#file399239line47>
> >
> > Even if I implement it this way won't I still have to cast arguments[0] to a PrimitiveObjectInspector in order to call getPrimitiveCategory()? If so then I don't see the advantage of doing it your way you suggest.
Well, that just gives a little clean code without try... catch... block. The benefit is marginal, but personally I think it's better to do explicit check than relying on runtime exception.
- Xuefu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30732
-----------------------------------------------------------
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30732
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java
<https://reviews.apache.org/r/15654/#comment58828>
Even if I implement it this way won't I still have to cast arguments[0] to a PrimitiveObjectInspector in order to call getPrimitiveCategory()? If so then I don't see the advantage of doing it your way you suggest.
- Mohammad Islam
On Dec. 18, 2013, 3:16 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2013, 3:16 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/
-----------------------------------------------------------
(Updated Dec. 18, 2013, 3:16 a.m.)
Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
Changes
-------
Includes Carl's comments of moving the Test* file to correct location.
Bugs: HIVE-5829
https://issues.apache.org/jira/browse/HIVE-5829
Repository: hive-git
Description
-------
Rewrite the UDFS *pads and *trim using GenericUDF.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java PRE-CREATION
Diff: https://reviews.apache.org/r/15654/diff/
Testing
-------
Thanks,
Mohammad Islam
Re: Review Request 15654: Rewrite Trim and Pad UDFs based on GenericUDF
Posted by Carl Steinbach <cw...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15654/#review30574
-----------------------------------------------------------
ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLTrim.java
<https://reviews.apache.org/r/15654/#comment58540>
For these new tests please change the package to org.apache.hive.ql.udf.generic and move them to the directory src/test/org/apache/hadoop/hive/ql/udf/generic.
- Carl Steinbach
On Dec. 17, 2013, midnight, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15654/
> -----------------------------------------------------------
>
> (Updated Dec. 17, 2013, midnight)
>
>
> Review request for hive, Ashutosh Chauhan, Carl Steinbach, and Jitendra Pandey.
>
>
> Bugs: HIVE-5829
> https://issues.apache.org/jira/browse/HIVE-5829
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite the UDFS *pads and *trim using GenericUDF.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java a895d65
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java bca1f26
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java dc00cf9
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java d1da19a
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java 2bcc5fa
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java 9652ce2
> ql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java 490886d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java eff251f
> ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFLpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFRTrim.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFRpad.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/TestGenericUDFTrim.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15654/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>