You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "tarak271 (via GitHub)" <gi...@apache.org> on 2023/09/01 15:28:10 UTC

[GitHub] [hive] tarak271 opened a new pull request, #4653: HIVE-27663 - implement array_position UDF in Hive

tarak271 opened a new pull request, #4653:
URL: https://github.com/apache/hive/pull/4653

   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
     2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
     3. Ensure you have added or run the appropriate tests for your PR: 
     4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX:  Your PR title ...'.
     5. Be sure to keep the PR description updated to reflect all changes.
     6. Please write your PR title to summarize what this PR proposes.
     7. If possible, provide a concise example to reproduce the issue for a faster review.
   
   -->
   
   ### What changes were proposed in this pull request?
   <!--
   Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. 
   If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
     1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
     2. If you fix some SQL features, you can provide some references of other DBMSes.
     3. If there is design documentation, please add the link.
     4. If there is a discussion in the mailing list, please add the link.
   -->
   
   Implement array_position function in Hive
   ### Why are the changes needed?
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you propose a new API, clarify the use case for a new API.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   This enhancement is already implemented in Spark
   ### Does this PR introduce _any_ user-facing change?
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description, screenshot and/or a reproducable example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released Hive versions or within the unreleased branches such as master.
   If no, write 'No'.
   -->
   No
   ### Is the change a dependency upgrade?
   <!--
   If yes, please attach a file with output from mvn dependency:tree to validate a complete upgrade of dependency.
   If no, write 'No'.
   -->
   No
   
   ### How was this patch tested?
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   -->
   Created Junit tests as well as qtests as part of this change


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1370110866


##########
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java:
##########
@@ -1263,6 +1267,33 @@ public static int compare(Object o1, ObjectInspector oi1, Object o2,
     }
   }
 
+  public static boolean isConversionSupported(ObjectInspector oi1, ObjectInspector oi2) {
+    if (oi1.getCategory() == oi2.getCategory()) {
+      if (oi1.getCategory() == Category.PRIMITIVE) {
+        PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+        PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+        if ((poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR)
+            && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)) || (
+            poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)
+                && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR))) {
+          return true;

Review Comment:
   What's the reason why we need this? I feel this is a little risky since it affect several places.



##########
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java:
##########
@@ -1263,6 +1267,33 @@ public static int compare(Object o1, ObjectInspector oi1, Object o2,
     }
   }
 
+  public static boolean isConversionSupported(ObjectInspector oi1, ObjectInspector oi2) {
+    if (oi1.getCategory() == oi2.getCategory()) {
+      if (oi1.getCategory() == Category.PRIMITIVE) {
+        PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+        PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+        if ((poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR)
+            && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)) || (
+            poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)
+                && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR))) {
+          return true;

Review Comment:
   What's the reason why we need this? I feel this is a little risky since it affects several places.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "saihemanth-cloudera (via GitHub)" <gi...@apache.org>.
saihemanth-cloudera merged PR #4653:
URL: https://github.com/apache/hive/pull/4653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1413142929


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -123,4 +127,55 @@ ObjectInspector initListOI(ObjectInspector[] arguments) {
         return ObjectInspectorFactory.getStandardListObjectInspector(initOI(arguments));
     }
 
+    void checkValueAndListElementTypes(ObjectInspector arrayElementOI, String functionName, ObjectInspector valueOI,
+        int elementIndex) throws UDFArgumentTypeException {
+        // Check if list element and value are of same type
+        if (!ObjectInspectorUtils.compareTypes(arrayElementOI, valueOI)) {
+            if (arrayElementOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || arrayElementOI.getTypeName()
+                .equals(serdeConstants.STRING_TYPE_NAME)) {
+                if (valueOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || valueOI.getTypeName()
+                    .equals(serdeConstants.STRING_TYPE_NAME)) {
+                    return;
+                }
+            }

Review Comment:
   I'm curious why we need to make string and varchar comparable here. I understand some types are comparable.
   
   ```
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select cast('abc' as string) == cast('abcd' as varchar(3));
   +-------+
   |  _c0  |
   +-------+
   | true  |
   +-------+
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select cast(2 as int) = cast(2 as bigint);
   +-------+
   |  _c0  |
   +-------+
   | true  |
   +-------+
   ```
   
   Is there a case where the string <-> varchar comparison gives us convenience?



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -123,4 +127,55 @@ ObjectInspector initListOI(ObjectInspector[] arguments) {
         return ObjectInspectorFactory.getStandardListObjectInspector(initOI(arguments));
     }
 
+    void checkValueAndListElementTypes(ObjectInspector arrayElementOI, String functionName, ObjectInspector valueOI,
+        int elementIndex) throws UDFArgumentTypeException {
+        // Check if list element and value are of same type
+        if (!ObjectInspectorUtils.compareTypes(arrayElementOI, valueOI)) {
+            if (arrayElementOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || arrayElementOI.getTypeName()
+                .equals(serdeConstants.STRING_TYPE_NAME)) {
+                if (valueOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || valueOI.getTypeName()
+                    .equals(serdeConstants.STRING_TYPE_NAME)) {
+                    return;
+                }
+            }
+            throw new UDFArgumentTypeException(elementIndex,
+                String.format("%s type element is expected at function %s(array<%s>,%s), but %s is found",
+                    arrayElementOI.getTypeName(), functionName, arrayElementOI.getTypeName(),
+                    arrayElementOI.getTypeName(), valueOI.getTypeName()));
+        }
+    }
+
+    boolean isConversionSupported(ObjectInspector oi1, ObjectInspector oi2) {
+        if (oi1.getCategory() == oi2.getCategory()) {
+            if (oi1.getCategory() == ObjectInspector.Category.PRIMITIVE) {
+                PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+                PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+                return (poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR)
+                    && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)) || (
+                    poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)
+                        && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR));
+            }
+        }
+        return false;
+    }
+
+    int compareConvertibleElements(ObjectInspector oi1, Object o1, ObjectInspector oi2, Object o2) {
+        PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+        PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+        if (String.valueOf(poi1.getPrimitiveWritableObject(o1))
+            .equals(String.valueOf(poi2.getPrimitiveWritableObject(o2)))) {
+            return 0;
+        } else {
+            return -1;
+        }
+    }
+
+    int compareElements(Object value, ObjectInspector valueOI, Object listElement) {
+        if ((listElement != null) && (ObjectInspectorUtils.compare(value, valueOI, listElement, arrayElementOI) == 0
+            || (isConversionSupported(valueOI, arrayElementOI)
+            && compareConvertibleElements(valueOI, value, arrayElementOI, listElement) == 0))) {
+            return 0;
+        }
+        return -1;

Review Comment:
   I think we should follow the convention of `Comparator#compare` if we return an integer here. If we really need only -1 and 0, the type of the return value may be boolean.



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -123,4 +127,55 @@ ObjectInspector initListOI(ObjectInspector[] arguments) {
         return ObjectInspectorFactory.getStandardListObjectInspector(initOI(arguments));
     }
 
+    void checkValueAndListElementTypes(ObjectInspector arrayElementOI, String functionName, ObjectInspector valueOI,
+        int elementIndex) throws UDFArgumentTypeException {
+        // Check if list element and value are of same type
+        if (!ObjectInspectorUtils.compareTypes(arrayElementOI, valueOI)) {
+            if (arrayElementOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || arrayElementOI.getTypeName()
+                .equals(serdeConstants.STRING_TYPE_NAME)) {
+                if (valueOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || valueOI.getTypeName()
+                    .equals(serdeConstants.STRING_TYPE_NAME)) {
+                    return;
+                }
+            }
+            throw new UDFArgumentTypeException(elementIndex,
+                String.format("%s type element is expected at function %s(array<%s>,%s), but %s is found",
+                    arrayElementOI.getTypeName(), functionName, arrayElementOI.getTypeName(),
+                    arrayElementOI.getTypeName(), valueOI.getTypeName()));
+        }
+    }
+
+    boolean isConversionSupported(ObjectInspector oi1, ObjectInspector oi2) {
+        if (oi1.getCategory() == oi2.getCategory()) {
+            if (oi1.getCategory() == ObjectInspector.Category.PRIMITIVE) {
+                PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+                PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+                return (poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR)
+                    && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)) || (
+                    poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)
+                        && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR));
+            }
+        }
+        return false;
+    }
+
+    int compareConvertibleElements(ObjectInspector oi1, Object o1, ObjectInspector oi2, Object o2) {
+        PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+        PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+        if (String.valueOf(poi1.getPrimitiveWritableObject(o1))
+            .equals(String.valueOf(poi2.getPrimitiveWritableObject(o2)))) {

Review Comment:
   I guess this should be private since this is applicable only when o1 and o2 are either string or varchar.



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -123,4 +127,55 @@ ObjectInspector initListOI(ObjectInspector[] arguments) {
         return ObjectInspectorFactory.getStandardListObjectInspector(initOI(arguments));
     }
 
+    void checkValueAndListElementTypes(ObjectInspector arrayElementOI, String functionName, ObjectInspector valueOI,
+        int elementIndex) throws UDFArgumentTypeException {
+        // Check if list element and value are of same type
+        if (!ObjectInspectorUtils.compareTypes(arrayElementOI, valueOI)) {
+            if (arrayElementOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || arrayElementOI.getTypeName()
+                .equals(serdeConstants.STRING_TYPE_NAME)) {
+                if (valueOI.getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME) || valueOI.getTypeName()
+                    .equals(serdeConstants.STRING_TYPE_NAME)) {
+                    return;
+                }
+            }
+            throw new UDFArgumentTypeException(elementIndex,
+                String.format("%s type element is expected at function %s(array<%s>,%s), but %s is found",
+                    arrayElementOI.getTypeName(), functionName, arrayElementOI.getTypeName(),
+                    arrayElementOI.getTypeName(), valueOI.getTypeName()));
+        }
+    }
+
+    boolean isConversionSupported(ObjectInspector oi1, ObjectInspector oi2) {
+        if (oi1.getCategory() == oi2.getCategory()) {
+            if (oi1.getCategory() == ObjectInspector.Category.PRIMITIVE) {
+                PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+                PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+                return (poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR)
+                    && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)) || (
+                    poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)
+                        && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR));
+            }
+        }
+        return false;
+    }
+
+    int compareConvertibleElements(ObjectInspector oi1, Object o1, ObjectInspector oi2, Object o2) {
+        PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+        PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+        if (String.valueOf(poi1.getPrimitiveWritableObject(o1))
+            .equals(String.valueOf(poi2.getPrimitiveWritableObject(o2)))) {

Review Comment:
   I may guess it is more robust to use a Converter. I think this is OK if we only support STRING <-> VARCHAR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1964195885

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&inNewCodePeriod=true)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&inNewCodePeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Duplication  
     
   [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1365040688


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -123,4 +124,15 @@ ObjectInspector initListOI(ObjectInspector[] arguments) {
         return ObjectInspectorFactory.getStandardListObjectInspector(initOI(arguments));
     }
 
+    void checkValueAndListElementTypes(ObjectInspector arrayElementOI, ObjectInspector valueOI, int elementIndex)
+        throws UDFArgumentTypeException {
+        // Check if list element and value are of same type
+        if (!ObjectInspectorUtils.compareTypes(arrayElementOI, valueOI)) {
+            throw new UDFArgumentTypeException(elementIndex,
+                String.format("%s type element is expected at function array_position(array<%s>,%s), but %s is found",

Review Comment:
   My bad, created placefolder for function name as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1769309404

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=duplicated_lines_density&view=list) No Duplication information
   
   ![warning](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning-16px.png 'warning') The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
   Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1830202070

   @okumin  could you pls help with review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1365041796


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+  private static final int ELEMENT_IDX = 1;
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arrayOI.getListElementObjectInspector(),arguments[ELEMENT_IDX],ELEMENT_IDX);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) < 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array);
+    // Handling Varchar type this way as Object comparison between string and varchar will not work
+    if ((argumentOIs[ELEMENT_IDX].getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME)

Review Comment:
   The below example is for a different UDF, but applies to this as well
   ```
   CREATE TABLE temp (top ARRAY<STRING>);
   insert into temp values(array(cast('true' as VARCHAR(10))));
   select array_contains(top,cast(true as VARCHAR(10))) from temp;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1774508489

    @okumin  @saihemanth-cloudera  need your help in review and merge


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #4653: HIVE-27663 - implement array_position UDF in Hive
URL: https://github.com/apache/hive/pull/4653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] tarak271 commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1316735497


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")

Review Comment:
   Yes, tried to replicate spark's behaviour



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1922541930

   This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
   Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #4653: HIVE-27663 - implement array_position UDF in Hive
URL: https://github.com/apache/hive/pull/4653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] tarak271 commented on pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1740555902

   Created https://issues.apache.org/jira/browse/HIVE-27681 to handle varchar data type in all other Array UDFs. @okumin @saihemanth-cloudera need your help in review and merge


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] okumin commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1341969626


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+  private static final int ELEMENT_IDX = 1;
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arrayOI.getListElementObjectInspector(),arguments[ELEMENT_IDX],ELEMENT_IDX);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) < 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array);
+    // Handling Varchar type this way as Object comparison between string and varchar will not work
+    if ((argumentOIs[ELEMENT_IDX].getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME)
+        || ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getListElementObjectInspector().getTypeName()
+        .contains(serdeConstants.VARCHAR_TYPE_NAME))) {
+      for (int index = 0; index < resultArray.size(); index++) {
+        if (resultArray.get(index).toString().equals(value.toString())) {
+          return new IntWritable(index + 1);
+        }
+      }
+    }
+    return new IntWritable(resultArray.indexOf(value) + 1);

Review Comment:
   We might have assumed the returned value of `ListObjectInspector#getList` can be directly used. But I started feeling it is still an abstract value, and then we need to parse them via the object inspector of the element.
   https://github.com/apache/hive/blob/rel/release-4.0.0-beta-1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayContains.java#L121-L122
   
   This is the example where I compared arrays in a table and arrays or an element as a literal. It could be better to check other similar UDFs that have been merged first.
   
   ```
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> create table test as select array('a', 'b', 'c', 'b') as a union all select array('a', 'c', 'd') as a;
   ...
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select a, array_contains(a, 'b') from test;
   +--------------------+--------+
   |         a          |  _c1   |
   +--------------------+--------+
   | ["a","b","c","b"]  | true   |
   | ["a","c","d"]      | false  |
   +--------------------+--------+
   ...
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select a, array_position(a, 'b') from test;
   +--------------------+------+
   |         a          | _c1  |
   +--------------------+------+
   | ["a","b","c","b"]  | 0    |
   | ["a","c","d"]      | 0    |
   +--------------------+------+
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select a, array_except(a, array('a', 'd')) from test;
   ...
   +--------------------+----------------+
   |         a          |      _c1       |
   +--------------------+----------------+
   | ["a","b","c","b"]  | ["a","b","c"]  |
   | ["a","c","d"]      | ["a","c","d"]  |
   +--------------------+----------------+
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] tarak271 commented on pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1707671332

   @okumin I have updated logic to deal with array of Varchar types. Will create another pull request with these varchar changes to the rest of array udfs


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1783168621

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=duplicated_lines_density&view=list) No Duplication information
   
   ![warning](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning-16px.png 'warning') The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
   Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1948915765

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&inNewCodePeriod=true)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&inNewCodePeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Duplication  
     
   [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] tarak271 commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1316736309


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -34,6 +35,7 @@
 public abstract class AbstractGenericUDFArrayBase extends GenericUDF {
 
     static final int ARRAY_IDX = 0;
+    static final int ELEMENT_IDX = 1;

Review Comment:
   Yes, keeping only checkValueAndListElementTypes(arrayElementOI, valueOI, elementIndex) method with index argument



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] okumin commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1314898179


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -34,6 +35,7 @@
 public abstract class AbstractGenericUDFArrayBase extends GenericUDF {
 
     static final int ARRAY_IDX = 0;
+    static final int ELEMENT_IDX = 1;

Review Comment:
   I guess it is more flexible if `checkValueAndListElementTypes` accepts an index than we hardcode the index here.
   For example, `array_insert` of Spark SQL receives an element in the 3rd argument, not the 2nd.
   https://spark.apache.org/docs/latest/api/sql/index.html#array_insert
   
   Another option is we keep only `checkValueAndListElementTypes(arrayElementOI, valueOI)`, removing `checkValueAndListElementTypes(arguments, index?)`.



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arguments);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) <= 0 || value == null) {

Review Comment:
   I guess the first condition should be `arrayOI.getListLength(array) < 0`.



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")

Review Comment:
   I originally thought arrays of Hive, Spark, MySQL, etc. were 0-based. But I also see `array_position` of Spark is 1-based...
   - https://spark.apache.org/docs/latest/api/sql/index.html#array_position
   
   ```
   spark-sql (default)> SELECT array(1, 2, 3, 4, 5)[3];
   4
   spark-sql (default)> SELECT array_position(array(1, 2, 3, 4, 5), 3);
   3
   ```



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arguments);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) <= 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = new ArrayList<>(((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array));

Review Comment:
   We may not need to copy the original object here because we don't return the reference of `resultArray`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] okumin commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1317316622


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -123,4 +124,15 @@ ObjectInspector initListOI(ObjectInspector[] arguments) {
         return ObjectInspectorFactory.getStandardListObjectInspector(initOI(arguments));
     }
 
+    void checkValueAndListElementTypes(ObjectInspector arrayElementOI, ObjectInspector valueOI, int elementIndex)
+        throws UDFArgumentTypeException {
+        // Check if list element and value are of same type
+        if (!ObjectInspectorUtils.compareTypes(arrayElementOI, valueOI)) {
+            throw new UDFArgumentTypeException(elementIndex,
+                String.format("%s type element is expected at function array_position(array<%s>,%s), but %s is found",

Review Comment:
   Sorry, I overlooked that this also contains `array_position(array<%s>, %s)`. Maybe, that part, at least the function name, has to be a placeholder.



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+  private static final int ELEMENT_IDX = 1;
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arrayOI.getListElementObjectInspector(),arguments[ELEMENT_IDX],ELEMENT_IDX);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) < 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array);
+    // Handling Varchar type this way as Object comparison between string and varchar will not work
+    if ((argumentOIs[ELEMENT_IDX].getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME)
+        || ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getListElementObjectInspector().getTypeName()
+        .contains(serdeConstants.VARCHAR_TYPE_NAME))) {
+      for (int index = 0; index < resultArray.size(); index++) {
+        if (resultArray.get(index).toString().equals(value.toString())) {
+          return new IntWritable(index + 1);
+        }
+      }
+    }
+    return new IntWritable(resultArray.indexOf(value) + 1);

Review Comment:
   We might have assumed the returned value of `ListObjectInspector#getList` can be directly used. But I started feeling it is still an abstract value, and then we need to parse them via the object inspector of the element.
   https://github.com/apache/hive/blob/rel/release-4.0.0-beta-1/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayContains.java#L121-L122
   
   This is the example where I compared arrays in a table and arrays or an element as a literal.
   
   ```
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> create table test as select array('a', 'b', 'c', 'b') as a union all select array('a', 'c', 'd') as a;
   ...
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select a, array_contains(a, 'b') from test;
   +--------------------+--------+
   |         a          |  _c1   |
   +--------------------+--------+
   | ["a","b","c","b"]  | true   |
   | ["a","c","d"]      | false  |
   +--------------------+--------+
   ...
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select a, array_position(a, 'b') from test;
   +--------------------+------+
   |         a          | _c1  |
   +--------------------+------+
   | ["a","b","c","b"]  | 0    |
   | ["a","c","d"]      | 0    |
   +--------------------+------+
   0: jdbc:hive2://hive-hiveserver2:10000/defaul> select a, array_except(a, array('a', 'd')) from test;
   ...
   +--------------------+----------------+
   |         a          |      _c1       |
   +--------------------+----------------+
   | ["a","b","c","b"]  | ["a","b","c"]  |
   | ["a","c","d"]      | ["a","c","d"]  |
   +--------------------+----------------+
   ```
   



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+  private static final int ELEMENT_IDX = 1;
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arrayOI.getListElementObjectInspector(),arguments[ELEMENT_IDX],ELEMENT_IDX);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) < 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array);
+    // Handling Varchar type this way as Object comparison between string and varchar will not work
+    if ((argumentOIs[ELEMENT_IDX].getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME)

Review Comment:
   What is examples when we need this one?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1365046503


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+  private static final int ELEMENT_IDX = 1;
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arrayOI.getListElementObjectInspector(),arguments[ELEMENT_IDX],ELEMENT_IDX);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) < 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array);
+    // Handling Varchar type this way as Object comparison between string and varchar will not work
+    if ((argumentOIs[ELEMENT_IDX].getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME)
+        || ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getListElementObjectInspector().getTypeName()
+        .contains(serdeConstants.VARCHAR_TYPE_NAME))) {
+      for (int index = 0; index < resultArray.size(); index++) {
+        if (resultArray.get(index).toString().equals(value.toString())) {
+          return new IntWritable(index + 1);
+        }
+      }
+    }
+    return new IntWritable(resultArray.indexOf(value) + 1);

Review Comment:
   Will fix the rest of the array UDFs in https://issues.apache.org/jira/browse/HIVE-27681



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1788462021

   @okumin could you pls help with review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] sonarcloud[bot] commented on pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1707789079

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=duplicated_lines_density&view=list) No Duplication information
   
   ![warning](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning-16px.png 'warning') The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
   Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #4653: HIVE-27663 - implement array_position UDF in Hive
URL: https://github.com/apache/hive/pull/4653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1374716288


##########
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java:
##########
@@ -1263,6 +1267,33 @@ public static int compare(Object o1, ObjectInspector oi1, Object o2,
     }
   }
 
+  public static boolean isConversionSupported(ObjectInspector oi1, ObjectInspector oi2) {
+    if (oi1.getCategory() == oi2.getCategory()) {
+      if (oi1.getCategory() == Category.PRIMITIVE) {
+        PrimitiveObjectInspector poi1 = ((PrimitiveObjectInspector) oi1);
+        PrimitiveObjectInspector poi2 = ((PrimitiveObjectInspector) oi2);
+        if ((poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR)
+            && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)) || (
+            poi1.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.STRING)
+                && poi2.getPrimitiveCategory().equals(PrimitiveObjectInspector.PrimitiveCategory.VARCHAR))) {
+          return true;

Review Comment:
   Varchar and string should be convertible and be able to compare. Moving this to base class of Array UDFs, AbstractGenericUDFArrayBase



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1978010460

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&inNewCodePeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=new_accepted_issues&view=list)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&inNewCodePeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Duplication  
     
   [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] HIVE-27663 - implement array_position UDF in Hive [hive]

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1365042606


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+  private static final int ELEMENT_IDX = 1;
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arrayOI.getListElementObjectInspector(),arguments[ELEMENT_IDX],ELEMENT_IDX);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) < 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array);
+    // Handling Varchar type this way as Object comparison between string and varchar will not work
+    if ((argumentOIs[ELEMENT_IDX].getTypeName().contains(serdeConstants.VARCHAR_TYPE_NAME)
+        || ((ListObjectInspector) argumentOIs[ARRAY_IDX]).getListElementObjectInspector().getTypeName()
+        .contains(serdeConstants.VARCHAR_TYPE_NAME))) {
+      for (int index = 0; index < resultArray.size(); index++) {
+        if (resultArray.get(index).toString().equals(value.toString())) {
+          return new IntWritable(index + 1);
+        }
+      }
+    }
+    return new IntWritable(resultArray.indexOf(value) + 1);

Review Comment:
   Understood. Changed to use ObjectInspector.compare 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] sonarcloud[bot] commented on pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4653:
URL: https://github.com/apache/hive/pull/4653#issuecomment-1703014110

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4653)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4653&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4653&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4653&metric=duplicated_lines_density&view=list) No Duplication information
   
   ![warning](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning-16px.png 'warning') The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
   Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] tarak271 commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1316736564


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arguments);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) <= 0 || value == null) {
+      return null;
+    }
+    List<?> resultArray = new ArrayList<>(((ListObjectInspector) argumentOIs[ARRAY_IDX]).getList(array));

Review Comment:
   Done



##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFArrayPosition.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.IntWritable;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * GenericUDFArrayPosition.
+ */
+@Description(name = "array_position", value = "_FUNC_(array, element) - Returns the position of the first occurrence of "
+    + "element in array. Array indexing starts at 1. If the element value is NULL, a NULL is returned.", extended =
+    "Example:\n" + "  > SELECT _FUNC_(array(1, 2, 3,4,2), 2) FROM src;\n" + "  2")
+public class GenericUDFArrayPosition extends AbstractGenericUDFArrayBase {
+  static final String FUNC_NAME = "ARRAY_POSITION";
+
+  public GenericUDFArrayPosition() {
+    super(FUNC_NAME, 2, 2, ObjectInspector.Category.PRIMITIVE);
+  }
+
+  @Override
+  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
+    super.initialize(arguments);
+    checkValueAndListElementTypes(arguments);
+    return PrimitiveObjectInspectorFactory.writableIntObjectInspector;
+  }
+
+  @Override
+  public Object evaluate(DeferredObject[] arguments) throws HiveException {
+    Object array = arguments[ARRAY_IDX].get();
+    Object value = arguments[ELEMENT_IDX].get();
+    if (arrayOI.getListLength(array) <= 0 || value == null) {

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] tarak271 commented on a diff in pull request #4653: HIVE-27663 - implement array_position UDF in Hive

Posted by "tarak271 (via GitHub)" <gi...@apache.org>.
tarak271 commented on code in PR #4653:
URL: https://github.com/apache/hive/pull/4653#discussion_r1316736309


##########
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFArrayBase.java:
##########
@@ -34,6 +35,7 @@
 public abstract class AbstractGenericUDFArrayBase extends GenericUDF {
 
     static final int ARRAY_IDX = 0;
+    static final int ELEMENT_IDX = 1;

Review Comment:
   Yes, keeping only checkValueAndListElementTypes(arrayElementOI, valueOI, elementIndex) method 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org