You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/15 15:40:02 UTC
[GitHub] [spark] beliefer opened a new pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
beliefer opened a new pull request #31198:
URL: https://github.com/apache/spark/pull/31198
### What changes were proposed in this pull request?
`GETBIT` is a bitwise expression function given an INTEGER value, returns the value of a bit at a specified position.
`GETBIT( <integer_expr>, <bit_position> )`
Examples
select getbit(11, 100), getbit(11, 3), getbit(11, 2), getbit(11, 1), getbit(11, 0);
GETBIT(11, 3) | GETBIT(11, 2) | GETBIT(11, 1) | GETBIT(11, 0)
-- | -- | -- | --
1 | 0 | 1 | 1
The mainstream database support this feature show below:
**Teradata**
https://docs.teradata.com/reader/kmuOwjp1zEYg98JsB8fu_A/PK1oV1b2jqvG~ohRnOro9w
**Impala**
https://docs.cloudera.com/runtime/7.2.0/impala-sql-reference/topics/impala-bit-functions.html#bit_functions__getbit
**Snowflake**
https://docs.snowflake.com/en/sql-reference/functions/getbit.html
**Yellowbrick**
https://www.yellowbrick.com/docs/2.2/ybd_sqlref/getbit.html
### Why are the changes needed?
GETBIT is very useful.
### Does this PR introduce _any_ user-facing change?
Yes. GETBIT is a new bitwise function.
### How was this patch tested?
Jenkins test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761177997
**[Test build #134113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134113/testReport)** for PR 31198 at commit [`5fc1bc2`](https://github.com/apache/spark/commit/5fc1bc24ed14f079c5eee09a31961e29ec0cdf5f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class BitwiseGet(left: Expression, right: Expression)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761083679
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38696/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762340995
**[Test build #134203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134203/testReport)** for PR 31198 at commit [`90c9907`](https://github.com/apache/spark/commit/90c9907956e8d6f15feef73b28be23243a2d012a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394
**[Test build #134317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134317/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559970957
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,60 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Any, pos: Int, size: Int): Byte = {
Review comment:
this is inefficient because of boxing. I'd suggest the following
```
def checkPosition(pos: Int, size: Int) = ...
...
override def nullSafeEval(target: Any, pos: Any): Any = {
val posInt = pos.asInstanceOf[Int]
BitwiseGetUtil.checkPosition(posInt, bitSize)
((target.asInstanceOf[Number].longValue >> posInt) & 1).toByte
}
override def doGenCode... {
...
BitwiseGetUtil.checkPosition($pos, $bitSize)
(byte)((((long) target) >> $pos) & 1)
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764665520
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134327/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763789716
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134273/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762486856
**[Test build #134205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134205/testReport)** for PR 31198 at commit [`240bf54`](https://github.com/apache/spark/commit/240bf541fff6afee8ef078a2e5df8d3930d10b70).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763744421
**[Test build #134284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134284/testReport)** for PR 31198 at commit [`280772c`](https://github.com/apache/spark/commit/280772c65b97195bb7624fe867af6cbb94295dec).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763734746
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38862/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559675127
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Long, pos: Int): Long = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+ } else if (java.lang.Long.SIZE <= pos) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+ "bit length of the target long")
+ }
+ (target >> pos) & 1
+ }
+}
+
+@ExpressionDescription(
+ usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_(11, 0);
+ 1
+ > SELECT _FUNC_(11, 2);
+ 0
+ """,
+ since = "3.2.0",
+ group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+ override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
+
+ override def dataType: DataType = LongType
Review comment:
We decided use byte type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763635596
**[Test build #134277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134277/testReport)** for PR 31198 at commit [`e3b8fc3`](https://github.com/apache/spark/commit/e3b8fc3148f825e5e690db98ecb12f96745a3117).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761023619
**[Test build #134113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134113/testReport)** for PR 31198 at commit [`5fc1bc2`](https://github.com/apache/spark/commit/5fc1bc24ed14f079c5eee09a31961e29ec0cdf5f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762420857
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38790/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762395834
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38788/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762395834
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38788/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762373599
**[Test build #134205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134205/testReport)** for PR 31198 at commit [`240bf54`](https://github.com/apache/spark/commit/240bf541fff6afee8ef078a2e5df8d3930d10b70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763631861
**[Test build #134273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134273/testReport)** for PR 31198 at commit [`c2f46c9`](https://github.com/apache/spark/commit/c2f46c9bd9fb991fc855982ec41e6fd0fa33a609).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762373599
**[Test build #134205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134205/testReport)** for PR 31198 at commit [`240bf54`](https://github.com/apache/spark/commit/240bf541fff6afee8ef078a2e5df8d3930d10b70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764517880
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764598620
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38914/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762458236
**[Test build #134203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134203/testReport)** for PR 31198 at commit [`90c9907`](https://github.com/apache/spark/commit/90c9907956e8d6f15feef73b28be23243a2d012a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561861375
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
*/
def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
+ /**
+ * Get the value of the bit (0 or 1) at the specified position.
Review comment:
let's copy the doc from the SQL expression.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763650907
**[Test build #134277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134277/testReport)** for PR 31198 at commit [`e3b8fc3`](https://github.com/apache/spark/commit/e3b8fc3148f825e5e690db98ecb12f96745a3117).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31198:
URL: https://github.com/apache/spark/pull/31198
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764499204
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38904/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761083679
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38696/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764497904
**[Test build #134317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134317/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r560972007
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
*/
def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
+ /**
+ * Get the value of the bit (0 or 1) at the specified position.
+ *
+ * @group bitwise_funcs
+ * @since 3.2.0
+ */
+ def bitwiseGet(e: Column, pos: Column): Column = withExpr { BitwiseGet(e.expr, pos.expr) }
Review comment:
`getBit`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764482693
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38904/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tanelk commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
tanelk commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559717112
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,60 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Any, pos: Int, size: Int): Byte = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+ } else if (size <= pos) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the bit upper limit")
+ }
+ val longValue = size match {
+ case java.lang.Byte.SIZE => target.asInstanceOf[Byte]
+ case java.lang.Short.SIZE => target.asInstanceOf[Short]
+ case java.lang.Integer.SIZE => target.asInstanceOf[Int]
+ case java.lang.Long.SIZE => target.asInstanceOf[Long]
+ }
+ ((longValue >> pos) & 1).byteValue
+ }
+}
+
+@ExpressionDescription(
+ usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_(11, 0);
+ 1
+ > SELECT _FUNC_(11, 2);
+ 0
+ """,
+ since = "3.2.0",
+ group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
Review comment:
Shouldn't this be `ExpectsInputTypes` instead of `ImplicitCastInputTypes`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559454256
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Long, pos: Int): Long = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+ } else if (java.lang.Long.SIZE <= pos) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+ "bit length of the target long")
+ }
+ (target >> pos) & 1
+ }
+}
+
+@ExpressionDescription(
+ usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_(11, 0);
+ 1
+ > SELECT _FUNC_(11, 2);
+ 0
+ """,
+ since = "3.2.0",
+ group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+ override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
+
+ override def dataType: DataType = LongType
Review comment:
Return type: Same as the input value
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764708706
**[Test build #134338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134338/testReport)** for PR 31198 at commit [`807783b`](https://github.com/apache/spark/commit/807783b4fc3543c98037ebe8bb4afb75be6af403).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764761961
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38925/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561861162
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BitwiseExpressionsSuite.scala
##########
@@ -18,6 +18,7 @@
package org.apache.spark.sql.catalyst.expressions
import org.apache.spark.SparkFunSuite
+ import org.apache.spark.sql.catalyst.dsl.expressions._
Review comment:
nit: remove extra spaces at the beginning
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764702781
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763734746
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38862/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763328941
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38834/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764531225
**[Test build #134327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134327/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762392793
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38790/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764870581
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134338/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763829861
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38870/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763338590
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38834/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764598620
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761065152
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38696/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763744421
**[Test build #134284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134284/testReport)** for PR 31198 at commit [`280772c`](https://github.com/apache/spark/commit/280772c65b97195bb7624fe867af6cbb94295dec).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764745040
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38925/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559383664
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Long, pos: Int): Long = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+ } else if (java.lang.Long.SIZE <= pos) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+ "bit length of the target long")
+ }
+ (target >> pos) & 1
+ }
+}
+
+@ExpressionDescription(
+ usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_(11, 0);
+ 1
+ > SELECT _FUNC_(11, 2);
+ 0
+ """,
+ since = "3.2.0",
+ group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+ override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
Review comment:
we can accept long type only, and the analyzer will cast int to long for us.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-765123562
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764517880
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r560968629
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,57 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def checkPosition(pos: Int, size: Int): Unit = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
Review comment:
`$pos is less than zero`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763785458
**[Test build #134273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134273/testReport)** for PR 31198 at commit [`c2f46c9`](https://github.com/apache/spark/commit/c2f46c9bd9fb991fc855982ec41e6fd0fa33a609).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764531225
**[Test build #134327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134327/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559383381
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Long, pos: Int): Long = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+ } else if (java.lang.Long.SIZE <= pos) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+ "bit length of the target long")
+ }
+ (target >> pos) & 1
+ }
+}
+
+@ExpressionDescription(
+ usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_(11, 0);
+ 1
+ > SELECT _FUNC_(11, 2);
+ 0
+ """,
+ since = "3.2.0",
+ group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+ override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
+
+ override def dataType: DataType = LongType
Review comment:
if it only returns 0 or 1, why we use long type? how about other databases?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762487543
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134205/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394
**[Test build #134317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134317/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764655840
**[Test build #134327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134327/testReport)** for PR 31198 at commit [`fbb8cb6`](https://github.com/apache/spark/commit/fbb8cb6881d6c80785e03b94a35c0ce27058fa0c).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763635596
**[Test build #134277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134277/testReport)** for PR 31198 at commit [`e3b8fc3`](https://github.com/apache/spark/commit/e3b8fc3148f825e5e690db98ecb12f96745a3117).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761547703
cc @wangyum
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561926531
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
*/
def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
+ /**
+ * Get the value of the bit (0 or 1) at the specified position.
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764462394
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762359996
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38788/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764524408
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763684119
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134277/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763716615
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38862/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763449566
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134249/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764552125
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38914/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764868698
**[Test build #134338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134338/testReport)** for PR 31198 at commit [`807783b`](https://github.com/apache/spark/commit/807783b4fc3543c98037ebe8bb4afb75be6af403).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-765123562
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r560972007
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
*/
def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
+ /**
+ * Get the value of the bit (0 or 1) at the specified position.
+ *
+ * @group bitwise_funcs
+ * @since 3.2.0
+ */
+ def bitwiseGet(e: Column, pos: Column): Column = withExpr { BitwiseGet(e.expr, pos.expr) }
Review comment:
`getBit`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762466974
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134203/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761023619
**[Test build #134113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134113/testReport)** for PR 31198 at commit [`5fc1bc2`](https://github.com/apache/spark/commit/5fc1bc24ed14f079c5eee09a31961e29ec0cdf5f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763835779
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134284/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763357281
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38834/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-765127020
@cloud-fan Thanks for your help!
@tanelk @srowen Thanks for your review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763835256
**[Test build #134284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134284/testReport)** for PR 31198 at commit [`280772c`](https://github.com/apache/spark/commit/280772c65b97195bb7624fe867af6cbb94295dec).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761187831
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134113/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763631861
**[Test build #134273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134273/testReport)** for PR 31198 at commit [`c2f46c9`](https://github.com/apache/spark/commit/c2f46c9bd9fb991fc855982ec41e6fd0fa33a609).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763809325
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38870/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764761961
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38925/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762466974
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134203/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561926531
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
*/
def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
+ /**
+ * Get the value of the bit (0 or 1) at the specified position.
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763357281
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38834/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762374017
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38788/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763684119
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134277/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763835779
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134284/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761187831
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134113/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763317778
**[Test build #134249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134249/testReport)** for PR 31198 at commit [`57e9cc5`](https://github.com/apache/spark/commit/57e9cc531d5053c8b3db48cf5f05b4726d78bc87).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764761921
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38925/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763317778
**[Test build #134249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134249/testReport)** for PR 31198 at commit [`57e9cc5`](https://github.com/apache/spark/commit/57e9cc531d5053c8b3db48cf5f05b4726d78bc87).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763829861
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38870/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763789716
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134273/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762487543
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134205/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r561861162
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/BitwiseExpressionsSuite.scala
##########
@@ -18,6 +18,7 @@
package org.apache.spark.sql.catalyst.expressions
import org.apache.spark.SparkFunSuite
+ import org.apache.spark.sql.catalyst.dsl.expressions._
Review comment:
nit: remove extra spaces at the beginning
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##########
@@ -1413,6 +1413,14 @@ object functions {
*/
def bitwiseNOT(e: Column): Column = withExpr { BitwiseNot(e.expr) }
+ /**
+ * Get the value of the bit (0 or 1) at the specified position.
Review comment:
let's copy the doc from the SQL expression.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762407911
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38790/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763430378
**[Test build #134249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134249/testReport)** for PR 31198 at commit [`57e9cc5`](https://github.com/apache/spark/commit/57e9cc531d5053c8b3db48cf5f05b4726d78bc87).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-761048226
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38696/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763684397
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38862/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764708706
**[Test build #134338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134338/testReport)** for PR 31198 at commit [`807783b`](https://github.com/apache/spark/commit/807783b4fc3543c98037ebe8bb4afb75be6af403).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31198:
URL: https://github.com/apache/spark/pull/31198
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31198:
URL: https://github.com/apache/spark/pull/31198#discussion_r559383664
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitwiseExpressions.scala
##########
@@ -204,3 +205,47 @@ case class BitwiseCount(child: Expression)
case LongType => java.lang.Long.bitCount(input.asInstanceOf[Long])
}
}
+
+object BitwiseGetUtil {
+ def calculate(target: Long, pos: Int): Long = {
+ if (pos < 0) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos less than zero")
+ } else if (java.lang.Long.SIZE <= pos) {
+ throw new IllegalArgumentException(s"Invalid bit position: $pos exceeds the " +
+ "bit length of the target long")
+ }
+ (target >> pos) & 1
+ }
+}
+
+@ExpressionDescription(
+ usage = "_FUNC_(expr, pos) - Returns the value of the bit (0 or 1) at the specified position.",
+ examples = """
+ Examples:
+ > SELECT _FUNC_(11, 0);
+ 1
+ > SELECT _FUNC_(11, 2);
+ 0
+ """,
+ since = "3.2.0",
+ group = "bitwise_funcs")
+case class BitwiseGet(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant {
+
+ override def inputTypes: Seq[IntegralType] = Seq(LongType, IntegerType)
Review comment:
we can accept long type only, and the analyzer will cast int to long for us.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763784502
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38870/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762340995
**[Test build #134203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134203/testReport)** for PR 31198 at commit [`90c9907`](https://github.com/apache/spark/commit/90c9907956e8d6f15feef73b28be23243a2d012a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764870581
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134338/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-762420857
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38790/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-763449566
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134249/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31198: [SPARK-33245][SQL] Add built-in UDF - GETBIT
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31198:
URL: https://github.com/apache/spark/pull/31198#issuecomment-764572744
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38914/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org