[jira] [Commented] (HIVE-19256) UDF which shapes the input data according to the specified schema

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527296#comment-16527296 ] 

Hive QA commented on HIVE-19256:
--------------------------------

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  5m 14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  5m 11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  5m 11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 34s{color} | {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12238/dev-support/hive-personality.sh |
| git revision | master / 5b2cbb5 |
| Default Java | 1.8.0_111 |
| modules | C: . U: . |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12238/yetus.txt |
| Powered by | Apache Yetus    http://yetus.apache.org |


This message was automatically generated.



> UDF which shapes the input data according to the specified schema
> -----------------------------------------------------------------
>
>                 Key: HIVE-19256
>                 URL: https://issues.apache.org/jira/browse/HIVE-19256
>             Project: Hive
>          Issue Type: New Feature
>          Components: Hive
>    Affects Versions: 3.1.0
>            Reporter: Ratandeep Ratti
>            Assignee: Ratandeep Ratti
>            Priority: Major
>             Fix For: 3.2.0
>
>         Attachments: HIVE-19256.patch, HIVE-19256_1.patch
>
>
> We use this UDF a lot in our org. This UDF takes an object and a Hive schema and make sure the output object matches the schema completely. In some respects it is similar to {{named
> _struct}} UDF which can be used to select columns from a struct, but it is more general since it can work not only on structs, but all Hive data types (expect union). Also the schema can provide certain valid type conversions (int -> double etc)
> One scenario where this is quite useful is making sure that the Hive view created with a specific schema will have columns which will always match that schema. In Hive today when a view is created, new nested columns from the underlying table can leak out from the view, even though the user never wanted this behavior. Note that this leaking of columns is only for nested columns and not for top level columns, so in that regard this behavior of Hive is inconsistent.
> Sample usage of the UDF
> {code}
> generic_project(col, "struct<a:array<struct<c:int,d:string>>>") // Returning data which matches the input schema. Here extra columns which are not part of the input will be removed
> generic_project(col, "struct<a:double>") //  If the input column had a struct with col a as int . It would type cast 'a' to double.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)