You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/15 15:40:02 UTC

[GitHub] [spark] beliefer opened a new pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

beliefer opened a new pull request #31198:
URL: https://github.com/apache/spark/pull/31198


   ### What changes were proposed in this pull request?
   `GETBIT` is a bitwise expression function given an INTEGER value, returns the value of a bit at a specified position.
   `GETBIT( <integer_expr>, <bit_position> )`
   
   Examples
   select getbit(11, 100), getbit(11, 3), getbit(11, 2), getbit(11, 1), getbit(11, 0);
   GETBIT(11, 3) | GETBIT(11, 2) | GETBIT(11, 1) | GETBIT(11, 0)
   -- | -- | -- | --
   1 | 0 | 1 | 1
   
   The mainstream database support this feature show below:
   
   **Teradata**
   https://docs.teradata.com/reader/kmuOwjp1zEYg98JsB8fu_A/PK1oV1b2jqvG~ohRnOro9w
   
   **Impala**
   https://docs.cloudera.com/runtime/7.2.0/impala-sql-reference/topics/impala-bit-functions.html#bit_functions__getbit
   
   **Snowflake**
   https://docs.snowflake.com/en/sql-reference/functions/getbit.html
   
   **Yellowbrick**
   https://www.yellowbrick.com/docs/2.2/ybd_sqlref/getbit.html
   
   
   ### Why are the changes needed?
   GETBIT is very useful.
   
   
   ### Does this PR introduce _any_ user-facing change?
   Yes. GETBIT is a new bitwise function.
   
   
   ### How was this patch tested?
   Jenkins test
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761177997


   **[Test build #134113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134113/testReport)** for PR 31198 at commit [`5fc1bc2`](https://github.com/apache/spark/commit/5fc1bc24ed14f079c5eee09a31961e29ec0cdf5f).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `case class BitwiseGet(left: Expression, right: Expression)`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761083679


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38696/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762340995


   **[Test build #134203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134203/testReport)** for PR 31198 at commit [`90c9907`](https://github.com/apache/spark/commit/90c9907956e8d6f15feef73b28be23243a2d012a).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394


   **[Test build #134317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134317/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559970957



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,60 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Any, pos: Int, size: Int): Byte = {

Review comment:
       this is inefficient because of boxing. I'd suggest the following
   ```
   def checkPosition(pos: Int, size: Int) = ...
   ...
   override def nullSafeEval(target: Any, pos: Any): Any = {
     val posInt = pos.asInstanceOf[Int]
     BitwiseGetUtil.checkPosition(posInt, bitSize)
     ((target.asInstanceOf[Number].longValue >> posInt) & 1).toByte
   }
   override def doGenCode... {
     ...
     BitwiseGetUtil.checkPosition($pos, $bitSize)
     (byte)((((long) target) >> $pos) & 1)
   }
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764665520


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134327/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763789716


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134273/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762486856


   **[Test build #134205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134205/testReport)** for PR 31198 at commit [`240bf54`](https://github.com/apache/spark/commit/240bf541fff6afee8ef078a2e5df8d3930d10b70).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763744421


   **[Test build #134284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134284/testReport)** for PR 31198 at commit [`280772c`](https://github.com/apache/spark/commit/280772c65b97195bb7624fe867af6cbb94295dec).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763734746


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38862/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559675127



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Long, pos: Int): Long = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+    } else if (java.lang.Long.SIZE <= pos) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+        "bit length of the target long")
+    }
+    (target >> pos) & 1
+  }
+}
+
+@ExpressionDescription(
+  usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+  examples = """
+    Examples:
+      > SELECT _FUNC_(11, 0);
+       1
+      > SELECT _FUNC_(11, 2);
+       0
+  """,
+  since = "3.2.0",
+  group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+  override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
+
+  override def dataType: DataType = LongType

Review comment:
       We decided use byte type.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763635596


   **[Test build #134277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134277/testReport)** for PR 31198 at commit [`e3b8fc3`](https://github.com/apache/spark/commit/e3b8fc3148f825e5e690db98ecb12f96745a3117).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761023619


   **[Test build #134113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134113/testReport)** for PR 31198 at commit [`5fc1bc2`](https://github.com/apache/spark/commit/5fc1bc24ed14f079c5eee09a31961e29ec0cdf5f).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762420857


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38790/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762395834


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38788/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762395834


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38788/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762373599


   **[Test build #134205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134205/testReport)** for PR 31198 at commit [`240bf54`](https://github.com/apache/spark/commit/240bf541fff6afee8ef078a2e5df8d3930d10b70).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763631861


   **[Test build #134273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134273/testReport)** for PR 31198 at commit [`c2f46c9`](https://github.com/apache/spark/commit/c2f46c9bd9fb991fc855982ec41e6fd0fa33a609).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762373599


   **[Test build #134205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134205/testReport)** for PR 31198 at commit [`240bf54`](https://github.com/apache/spark/commit/240bf541fff6afee8ef078a2e5df8d3930d10b70).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764517880


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764598620


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38914/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762458236


   **[Test build #134203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134203/testReport)** for PR 31198 at commit [`90c9907`](https://github.com/apache/spark/commit/90c9907956e8d6f15feef73b28be23243a2d012a).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561861375



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
    */
   def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
 
+  /**
+   * Get the value of the bit (0 or 1) at the specified position.

Review comment:
       let's copy the doc from the SQL expression.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763650907


   **[Test build #134277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134277/testReport)** for PR 31198 at commit [`e3b8fc3`](https://github.com/apache/spark/commit/e3b8fc3148f825e5e690db98ecb12f96745a3117).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan closed pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31198:
URL: https://github.com/apache/spark/pull/31198


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764499204


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38904/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761083679


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38696/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764497904


   **[Test build #134317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134317/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r560972007



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
    */
   def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
 
+  /**
+   * Get the value of the bit (0 or 1) at the specified position.
+   *
+   * @group bitwise_funcs
+   * @since 3.2.0
+   */
+  def bitwiseGet(e: Column, pos: Column): Column = withExpr { BitwiseGet(e.expr, pos.expr) }

Review comment:
       `getBit`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764482693


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38904/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] tanelk commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
tanelk commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559717112



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,60 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Any, pos: Int, size: Int): Byte = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+    } else if (size <= pos) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the bit upper limit")
+    }
+    val longValue = size match {
+      case java.lang.Byte.SIZE => target.asInstanceOf[Byte]
+      case java.lang.Short.SIZE => target.asInstanceOf[Short]
+      case java.lang.Integer.SIZE => target.asInstanceOf[Int]
+      case java.lang.Long.SIZE => target.asInstanceOf[Long]
+    }
+    ((longValue >> pos) & 1).byteValue
+  }
+}
+
+@ExpressionDescription(
+  usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+  examples = """
+    Examples:
+      > SELECT _FUNC_(11, 0);
+       1
+      > SELECT _FUNC_(11, 2);
+       0
+  """,
+  since = "3.2.0",
+  group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {

Review comment:
       Shouldn't this be `ExpectsInputTypes` instead of `ImplicitCastInputTypes`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559454256



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Long, pos: Int): Long = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+    } else if (java.lang.Long.SIZE <= pos) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+        "bit length of the target long")
+    }
+    (target >> pos) & 1
+  }
+}
+
+@ExpressionDescription(
+  usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+  examples = """
+    Examples:
+      > SELECT _FUNC_(11, 0);
+       1
+      > SELECT _FUNC_(11, 2);
+       0
+  """,
+  since = "3.2.0",
+  group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+  override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
+
+  override def dataType: DataType = LongType

Review comment:
       Return type: Same as the input value




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764708706


   **[Test build #134338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134338/testReport)** for PR 31198 at commit [`807783b`](https://github.com/apache/spark/commit/807783b4fc3543c98037ebe8bb4afb75be6af403).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764761961


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38925/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561861162



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BitwiseExpressionsSuite.scala
##########
@@ -18,6 +18,7 @@
 package org.apache.spark.sql.catalyst.expressions
 
 import org.apache.spark.SparkFunSuite
+ import org.apache.spark.sql.catalyst.dsl.expressions._

Review comment:
       nit: remove extra spaces at the beginning




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764702781


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763734746


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38862/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763328941


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38834/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764531225


   **[Test build #134327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134327/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762392793


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38790/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764870581


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134338/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763829861


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38870/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763338590


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38834/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764598620






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761065152


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38696/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763744421


   **[Test build #134284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134284/testReport)** for PR 31198 at commit [`280772c`](https://github.com/apache/spark/commit/280772c65b97195bb7624fe867af6cbb94295dec).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764745040


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38925/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559383664



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Long, pos: Int): Long = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+    } else if (java.lang.Long.SIZE <= pos) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+        "bit length of the target long")
+    }
+    (target >> pos) & 1
+  }
+}
+
+@ExpressionDescription(
+  usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+  examples = """
+    Examples:
+      > SELECT _FUNC_(11, 0);
+       1
+      > SELECT _FUNC_(11, 2);
+       0
+  """,
+  since = "3.2.0",
+  group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+  override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)

Review comment:
       we can accept long type only, and the analyzer will cast int to long for us.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-765123562


   thanks, merging to master!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764517880






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r560968629



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,57 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def checkPosition(pos: Int, size: Int): Unit = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")

Review comment:
       `$pos is less than zero`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763785458


   **[Test build #134273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134273/testReport)** for PR 31198 at commit [`c2f46c9`](https://github.com/apache/spark/commit/c2f46c9bd9fb991fc855982ec41e6fd0fa33a609).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764531225


   **[Test build #134327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134327/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559383381



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Long, pos: Int): Long = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+    } else if (java.lang.Long.SIZE <= pos) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+        "bit length of the target long")
+    }
+    (target >> pos) & 1
+  }
+}
+
+@ExpressionDescription(
+  usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+  examples = """
+    Examples:
+      > SELECT _FUNC_(11, 0);
+       1
+      > SELECT _FUNC_(11, 2);
+       0
+  """,
+  since = "3.2.0",
+  group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+  override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
+
+  override def dataType: DataType = LongType

Review comment:
       if it only returns 0 or 1, why we use long type? how about other databases?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762487543


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134205/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394


   **[Test build #134317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134317/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764655840


   **[Test build #134327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134327/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763635596


   **[Test build #134277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134277/testReport)** for PR 31198 at commit [`e3b8fc3`](https://github.com/apache/spark/commit/e3b8fc3148f825e5e690db98ecb12f96745a3117).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761547703


   cc @wangyum 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561926531



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
    */
   def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
 
+  /**
+   * Get the value of the bit (0 or 1) at the specified position.

Review comment:
       OK




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762359996


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38788/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763684119


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134277/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763716615


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38862/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763449566


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134249/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764552125


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38914/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764868698


   **[Test build #134338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134338/testReport)** for PR 31198 at commit [`807783b`](https://github.com/apache/spark/commit/807783b4fc3543c98037ebe8bb4afb75be6af403).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-765123562


   thanks, merging to master!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r560972007



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
    */
   def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
 
+  /**
+   * Get the value of the bit (0 or 1) at the specified position.
+   *
+   * @group bitwise_funcs
+   * @since 3.2.0
+   */
+  def bitwiseGet(e: Column, pos: Column): Column = withExpr { BitwiseGet(e.expr, pos.expr) }

Review comment:
       `getBit`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762466974


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134203/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761023619


   **[Test build #134113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134113/testReport)** for PR 31198 at commit [`5fc1bc2`](https://github.com/apache/spark/commit/5fc1bc24ed14f079c5eee09a31961e29ec0cdf5f).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763835779


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134284/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763357281


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38834/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-765127020


   @cloud-fan Thanks for your help!
   @tanelk @srowen  Thanks for your review.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763835256


   **[Test build #134284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134284/testReport)** for PR 31198 at commit [`280772c`](https://github.com/apache/spark/commit/280772c65b97195bb7624fe867af6cbb94295dec).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761187831


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134113/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763631861


   **[Test build #134273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134273/testReport)** for PR 31198 at commit [`c2f46c9`](https://github.com/apache/spark/commit/c2f46c9bd9fb991fc855982ec41e6fd0fa33a609).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763809325


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38870/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764761961


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38925/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762466974


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134203/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561926531



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
    */
   def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
 
+  /**
+   * Get the value of the bit (0 or 1) at the specified position.

Review comment:
       OK




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763357281


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38834/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762374017


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38788/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763684119


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134277/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763835779


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134284/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761187831


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134113/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763317778


   **[Test build #134249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134249/testReport)** for PR 31198 at commit [`57e9cc5`](https://github.com/apache/spark/commit/57e9cc531d5053c8b3db48cf5f05b4726d78bc87).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764761921


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38925/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763317778


   **[Test build #134249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134249/testReport)** for PR 31198 at commit [`57e9cc5`](https://github.com/apache/spark/commit/57e9cc531d5053c8b3db48cf5f05b4726d78bc87).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763829861


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38870/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763789716


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134273/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762487543


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134205/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561861162



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BitwiseExpressionsSuite.scala
##########
@@ -18,6 +18,7 @@
 package org.apache.spark.sql.catalyst.expressions
 
 import org.apache.spark.SparkFunSuite
+ import org.apache.spark.sql.catalyst.dsl.expressions._

Review comment:
       nit: remove extra spaces at the beginning

##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
    */
   def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
 
+  /**
+   * Get the value of the bit (0 or 1) at the specified position.

Review comment:
       let's copy the doc from the SQL expression.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762407911


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38790/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763430378


   **[Test build #134249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134249/testReport)** for PR 31198 at commit [`57e9cc5`](https://github.com/apache/spark/commit/57e9cc531d5053c8b3db48cf5f05b4726d78bc87).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761048226


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38696/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763684397


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38862/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764708706


   **[Test build #134338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134338/testReport)** for PR 31198 at commit [`807783b`](https://github.com/apache/spark/commit/807783b4fc3543c98037ebe8bb4afb75be6af403).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan closed pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31198:
URL: https://github.com/apache/spark/pull/31198


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559383664



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
     case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
   }
 }
+
+object BitwiseGetUtil {
+  def calculate(target: Long, pos: Int): Long = {
+    if (pos < 0) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+    } else if (java.lang.Long.SIZE <= pos) {
+      throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+        "bit length of the target long")
+    }
+    (target >> pos) & 1
+  }
+}
+
+@ExpressionDescription(
+  usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+  examples = """
+    Examples:
+      > SELECT _FUNC_(11, 0);
+       1
+      > SELECT _FUNC_(11, 2);
+       0
+  """,
+  since = "3.2.0",
+  group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+  override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)

Review comment:
       we can accept long type only, and the analyzer will cast int to long for us.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763784502


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38870/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762340995


   **[Test build #134203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134203/testReport)** for PR 31198 at commit [`90c9907`](https://github.com/apache/spark/commit/90c9907956e8d6f15feef73b28be23243a2d012a).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764870581


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134338/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762420857


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38790/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763449566


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134249/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764572744


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38914/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org