You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/06/07 06:23:27 UTC
[GitHub] [spark] sarutak opened a new pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
sarutak opened a new pull request #32801:
URL: https://github.com/apache/spark/pull/32801
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This PR adds AES encryption/decryption builtin functions for SQL called `aes_encrypt` and `aes_decrypt`.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
Like hash functions, encryption/decryption is common for data processing.
Hive also implements these functions.
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
Yes. New functions will be available.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
New tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862021191
**[Test build #139836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139836/testReport)** for PR 32801 at commit [`996f787`](https://github.com/apache/spark/commit/996f787921860d002e275fe530af6b2ab429cf7e).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r728296512
##########
File path: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions;
+
+import org.apache.spark.sql.errors.QueryExecutionErrors;
+
+import javax.crypto.Cipher;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.GeneralSecurityException;
+
+/**
+ * An utility class for constructing expressions.
+ */
+public class ExpressionImplUtils {
+ public static byte[] aesEncrypt(byte[] input, byte[] key) {
+ return aesInternal(input, key, Cipher.ENCRYPT_MODE);
+ }
+
+ public static byte[] aesDecrypt(byte[] input, byte[] key) {
+ return aesInternal(input, key, Cipher.DECRYPT_MODE);
+ }
+
+ private static byte[] aesInternal(byte[] input, byte[] key, int mode) {
+ int inputLength = input.length;
+ int keyLength = key.length;
+ SecretKeySpec secretKey;
+
+ switch (keyLength) {
+ case 16:
+ case 24:
+ case 32:
+ secretKey = new SecretKeySpec(key, 0, keyLength, "AES");
+ break;
+ default:
+ throw QueryExecutionErrors.invalidAesKeyLengthError(keyLength);
+ }
+
+ try {
+ Cipher cipher = Cipher.getInstance("AES");
+ cipher.init(mode, secretKey);
+ return cipher.doFinal(input, 0, inputLength);
+ } catch (GeneralSecurityException e) {
+ throw new RuntimeException(e);
Review comment:
`GeneralSecurityException` should be wrapped by `RuntimeException` because it's a checked exception. Otherwise, compilation for codegen fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938364202
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144005/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r727820746
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +306,105 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The concrete implementation for AES encryption and decryption.
+ */
+object AesImpl {
Review comment:
Can we write it in java? e.g. `class ExpressionImplUtils` under `org.apache.spark.sql.catalyst.expressions.codegen`, and we can add more static methods in the future when we add new expressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942044056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r727820746
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +306,105 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The concrete implementation for AES encryption and decryption.
+ */
+object AesImpl {
Review comment:
Can we write it in java? e.g. `class ExpressionCodegenImpl` under `org.apache.spark.sql.catalyst.expressions.codegen`, and we can add more static methods in the future when we add new expressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942046204
**[Test build #144192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144192/testReport)** for PR 32801 at commit [`481791a`](https://github.com/apache/spark/commit/481791aad0e8831e0d2e67eec50f4920aca9e023).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942699118
**[Test build #144206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144206/testReport)** for PR 32801 at commit [`a1942ee`](https://github.com/apache/spark/commit/a1942ee9b73e835f1157bf99e9316f56b58488b7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `public class ExpressionImplUtils `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862019802
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44360/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862047380
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44364/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855885699
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139405/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-937368209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r727820746
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +306,105 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The concrete implementation for AES encryption and decryption.
+ */
+object AesImpl {
Review comment:
Can we write it in java? e.g. `class ExpressionImplUtils` under `org.apache.spark.sql.catalyst.expressions`, and we can add more static methods in the future when we add new expressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-941956726
**[Test build #144186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144186/testReport)** for PR 32801 at commit [`4559614`](https://github.com/apache/spark/commit/45596140808caf92159053903b8c64646728554b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-941956726
**[Test build #144186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144186/testReport)** for PR 32801 at commit [`4559614`](https://github.com/apache/spark/commit/45596140808caf92159053903b8c64646728554b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942191312
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144186/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942764795
**[Test build #144208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144208/testReport)** for PR 32801 at commit [`29cb68f`](https://github.com/apache/spark/commit/29cb68f92ea893e910816dbf348343170562f9b5).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class HistogramPlotBase(NumericPlotBase):`
* `class KdePlotBase(NumericPlotBase):`
* ` new_class = type(NameTypeHolder.short_name, (NameTypeHolder,), `
* `class Database(NamedTuple):`
* `class Table(NamedTuple):`
* `class Column(NamedTuple):`
* `class Function(NamedTuple):`
* ` protected class YarnSchedulerEndpoint(override val rpcEnv: RpcEnv)`
* `class IndexAlreadyExistsException(message: String, cause: Option[Throwable] = None)`
* `class NoSuchIndexException(message: String, cause: Option[Throwable] = None)`
* `case class SetCatalogAndNamespace(child: LogicalPlan) extends UnaryCommand `
* `case class CreateTempFunction(`
* `case class CreateFunction(`
* `case class CreateView(`
* `case class SetNamespaceCommand(namespace: Seq[String]) extends LeafRunnableCommand `
* `case class HashedRelationBroadcastMode(key: Seq[Expression], isNullAware: Boolean = false)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943001707
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144236/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862174910
**[Test build #139836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139836/testReport)** for PR 32801 at commit [`996f787`](https://github.com/apache/spark/commit/996f787921860d002e275fe530af6b2ab429cf7e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r728296512
##########
File path: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions;
+
+import org.apache.spark.sql.errors.QueryExecutionErrors;
+
+import javax.crypto.Cipher;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.GeneralSecurityException;
+
+/**
+ * An utility class for constructing expressions.
+ */
+public class ExpressionImplUtils {
+ public static byte[] aesEncrypt(byte[] input, byte[] key) {
+ return aesInternal(input, key, Cipher.ENCRYPT_MODE);
+ }
+
+ public static byte[] aesDecrypt(byte[] input, byte[] key) {
+ return aesInternal(input, key, Cipher.DECRYPT_MODE);
+ }
+
+ private static byte[] aesInternal(byte[] input, byte[] key, int mode) {
+ int inputLength = input.length;
+ int keyLength = key.length;
+ SecretKeySpec secretKey;
+
+ switch (keyLength) {
+ case 16:
+ case 24:
+ case 32:
+ secretKey = new SecretKeySpec(key, 0, keyLength, "AES");
+ break;
+ default:
+ throw QueryExecutionErrors.invalidAesKeyLengthError(keyLength);
+ }
+
+ try {
+ Cipher cipher = Cipher.getInstance("AES");
+ cipher.init(mode, secretKey);
+ return cipher.doFinal(input, 0, inputLength);
+ } catch (GeneralSecurityException e) {
+ throw new RuntimeException(e);
Review comment:
`GeneralSecurityException` should be wrapped by `RuntimeException` because it's a checked exception. Otherwise, compilation for codegen fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855772784
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43927/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r726767859
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +305,137 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The base implementation for AES encryption and decryption.
+ */
+abstract class AesBase(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+
+ override def dataType: DataType = BinaryType
+ override def nullable: Boolean = true
+ override def inputTypes: Seq[DataType] = Seq(BinaryType, BinaryType)
+ protected val cipherMode: Int
+
+ @transient lazy protected val cipher: Cipher = try {
+ Cipher.getInstance("AES")
+ } catch {
+ case e @ (_: NoSuchPaddingException | _: NoSuchAlgorithmException) =>
+ throw new RuntimeException(e)
+ }
+
+ protected override def nullSafeEval(input1: Any, input2: Any): Any = {
+ val input = input1.asInstanceOf[Array[Byte]]
+ val key = input2.asInstanceOf[Array[Byte]]
+ val inputLength = input.length
+ val keyLength = key.length
+ val secretKey = keyLength match {
+ case 16 | 24 | 32 => new SecretKeySpec(key, 0, keyLength, "AES")
+ case _ => null
+ }
+
+ if (secretKey == null) {
+ return null
+ }
+
+ try {
+ cipher.init(cipherMode, secretKey)
+ cipher.doFinal(input, 0, inputLength)
+ } catch {
+ case _: GeneralSecurityException =>
+ null
+ }
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
Review comment:
Seems better. I'll try it. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943122829
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48720/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943217323
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144240/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862021191
**[Test build #139836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139836/testReport)** for PR 32801 at commit [`996f787`](https://github.com/apache/spark/commit/996f787921860d002e275fe530af6b2ab429cf7e).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855623854
**[Test build #139400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139400/testReport)** for PR 32801 at commit [`eb7d86f`](https://github.com/apache/spark/commit/eb7d86f1b1872618949d86fac3c049515104ed58).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855748503
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43927/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942178552
**[Test build #144186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144186/testReport)** for PR 32801 at commit [`4559614`](https://github.com/apache/spark/commit/45596140808caf92159053903b8c64646728554b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class AesEncrypt(input: Expression, key: Expression, child: Expression)`
* `case class AesDecrypt(input: Expression, key: Expression, child: Expression)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938284951
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48482/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855802447
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43930/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862078024
**[Test build #139831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139831/testReport)** for PR 32801 at commit [`820cfac`](https://github.com/apache/spark/commit/820cfac03bfbde18100b1092dde74ebddf2b5f1c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942143226
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48671/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942412978
**[Test build #144206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144206/testReport)** for PR 32801 at commit [`a1942ee`](https://github.com/apache/spark/commit/a1942ee9b73e835f1157bf99e9316f56b58488b7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943217323
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144240/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943073482
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48720/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942139648
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48671/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-869274420
@dongjoon-hyun Thank you for letting me know. I didn't notice the previous PR. I'll check it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862079173
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139831/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862007710
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44360/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855695504
**[Test build #139400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139400/testReport)** for PR 32801 at commit [`eb7d86f`](https://github.com/apache/spark/commit/eb7d86f1b1872618949d86fac3c049515104ed58).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `abstract class AesBase(left: Expression, right: Expression)`
* `case class AesEncrypt(left: Expression, right: Expression) extends AesBase(left, right) `
* `case class AesDecrypt(left: Expression, right: Expression) extends AesBase(left, right) `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942293003
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144192/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942412978
**[Test build #144206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144206/testReport)** for PR 32801 at commit [`a1942ee`](https://github.com/apache/spark/commit/a1942ee9b73e835f1157bf99e9316f56b58488b7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942460049
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48685/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942284692
**[Test build #144192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144192/testReport)** for PR 32801 at commit [`481791a`](https://github.com/apache/spark/commit/481791aad0e8831e0d2e67eec50f4920aca9e023).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942143226
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48671/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942044056
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48665/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942538218
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48685/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942636094
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48687/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943003168
**[Test build #144240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144240/testReport)** for PR 32801 at commit [`bcb724c`](https://github.com/apache/spark/commit/bcb724c99eba1c76a51b984cb41021a3126a6105).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-937368463
I suspect it was reverted because it doesn't make sense to have them as DSL APIs, and at that time we have matched DSL and SQL functions before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #32801:
URL: https://github.com/apache/spark/pull/32801
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r659262486
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
##########
@@ -622,6 +622,8 @@ object FunctionRegistry {
expression[Sha1]("sha", true),
expression[Sha1]("sha1"),
expression[Sha2]("sha2"),
+ expression[AesEncrypt]("aes_encrypt"),
+ expression[AesDecrypt]("aes_decrypt"),
Review comment:
This seems to be supported by MySQL and Oracle, right? Could you mention it in the PR description?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855885430
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855716549
**[Test build #139408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139408/testReport)** for PR 32801 at commit [`7bbbb48`](https://github.com/apache/spark/commit/7bbbb48283f8cf061967cdee7ced4fdaf380de94).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855751758
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43930/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862186554
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139836/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-937368586
cc @gatorsmile and @cloud-fan FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942636094
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48687/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942726004
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144206/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943016000
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48716/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943001707
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144236/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943016000
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48716/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-937337147
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-937337147
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942017284
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48665/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943015885
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48716/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942961710
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48716/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938256137
**[Test build #144005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144005/testReport)** for PR 32801 at commit [`82443ef`](https://github.com/apache/spark/commit/82443ef5f860b6591ace1008616cf09c253dd744).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938296716
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48482/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r724883083
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +305,137 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The base implementation for AES encryption and decryption.
+ */
+abstract class AesBase(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+
+ override def dataType: DataType = BinaryType
+ override def nullable: Boolean = true
+ override def inputTypes: Seq[DataType] = Seq(BinaryType, BinaryType)
+ protected val cipherMode: Int
+
+ @transient lazy protected val cipher: Cipher = try {
+ Cipher.getInstance("AES")
+ } catch {
+ case e @ (_: NoSuchPaddingException | _: NoSuchAlgorithmException) =>
+ throw new RuntimeException(e)
+ }
+
+ protected override def nullSafeEval(input1: Any, input2: Any): Any = {
+ val input = input1.asInstanceOf[Array[Byte]]
+ val key = input2.asInstanceOf[Array[Byte]]
+ val inputLength = input.length
+ val keyLength = key.length
+ val secretKey = keyLength match {
+ case 16 | 24 | 32 => new SecretKeySpec(key, 0, keyLength, "AES")
+ case _ => null
+ }
+
+ if (secretKey == null) {
+ return null
+ }
+
+ try {
+ cipher.init(cipherMode, secretKey)
+ cipher.doFinal(input, 0, inputLength)
+ } catch {
+ case _: GeneralSecurityException =>
+ null
+ }
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
Review comment:
Today, I think it's more convenient to implement the expression logic with Java, and call the java method with `StaticInvoke` wrapped by `RuntimeReplaceable`. It's also easier to write UT for the Java methods, and we don't need to implement codegen by hand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-941956726
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938270370
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48482/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862047355
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44364/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942622048
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48687/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855772815
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43927/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855713227
**[Test build #139405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139405/testReport)** for PR 32801 at commit [`b65326c`](https://github.com/apache/spark/commit/b65326c0896018fe592486b77f5b6232a038a597).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r659413986
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
##########
@@ -622,6 +622,8 @@ object FunctionRegistry {
expression[Sha1]("sha", true),
expression[Sha1]("sha1"),
expression[Sha2]("sha2"),
+ expression[AesEncrypt]("aes_encrypt"),
+ expression[AesDecrypt]("aes_decrypt"),
Review comment:
Thanks. I've updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855716549
**[Test build #139408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139408/testReport)** for PR 32801 at commit [`7bbbb48`](https://github.com/apache/spark/commit/7bbbb48283f8cf061967cdee7ced4fdaf380de94).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r724883083
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +305,137 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The base implementation for AES encryption and decryption.
+ */
+abstract class AesBase(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+
+ override def dataType: DataType = BinaryType
+ override def nullable: Boolean = true
+ override def inputTypes: Seq[DataType] = Seq(BinaryType, BinaryType)
+ protected val cipherMode: Int
+
+ @transient lazy protected val cipher: Cipher = try {
+ Cipher.getInstance("AES")
+ } catch {
+ case e @ (_: NoSuchPaddingException | _: NoSuchAlgorithmException) =>
+ throw new RuntimeException(e)
+ }
+
+ protected override def nullSafeEval(input1: Any, input2: Any): Any = {
+ val input = input1.asInstanceOf[Array[Byte]]
+ val key = input2.asInstanceOf[Array[Byte]]
+ val inputLength = input.length
+ val keyLength = key.length
+ val secretKey = keyLength match {
+ case 16 | 24 | 32 => new SecretKeySpec(key, 0, keyLength, "AES")
+ case _ => null
+ }
+
+ if (secretKey == null) {
+ return null
+ }
+
+ try {
+ cipher.init(cipherMode, secretKey)
+ cipher.doFinal(input, 0, inputLength)
+ } catch {
+ case _: GeneralSecurityException =>
+ null
+ }
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
Review comment:
Today, I think it's more convenient to implement the expression logic with Java, and call the java method with `StaticInvoke` wrapped by `RuntimeReplaceable`. It's also easier to write UT for the Java methods if needed, and we don't need to implement codegen by hand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r724880473
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +305,137 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The base implementation for AES encryption and decryption.
+ */
+abstract class AesBase(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+
+ override def dataType: DataType = BinaryType
+ override def nullable: Boolean = true
+ override def inputTypes: Seq[DataType] = Seq(BinaryType, BinaryType)
+ protected val cipherMode: Int
+
+ @transient lazy protected val cipher: Cipher = try {
+ Cipher.getInstance("AES")
+ } catch {
+ case e @ (_: NoSuchPaddingException | _: NoSuchAlgorithmException) =>
+ throw new RuntimeException(e)
+ }
+
+ protected override def nullSafeEval(input1: Any, input2: Any): Any = {
+ val input = input1.asInstanceOf[Array[Byte]]
+ val key = input2.asInstanceOf[Array[Byte]]
+ val inputLength = input.length
+ val keyLength = key.length
+ val secretKey = keyLength match {
+ case 16 | 24 | 32 => new SecretKeySpec(key, 0, keyLength, "AES")
+ case _ => null
Review comment:
shall we fail instead of returning null to be more ANSI style?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938359208
**[Test build #144005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144005/testReport)** for PR 32801 at commit [`82443ef`](https://github.com/apache/spark/commit/82443ef5f860b6591ace1008616cf09c253dd744).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862035784
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44364/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855802447
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43930/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r727824214
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
##########
@@ -1826,4 +1826,9 @@ object QueryExecutionErrors {
def pivotNotAfterGroupByUnsupportedError(): Throwable = {
new UnsupportedOperationException("pivot is only supported after a groupBy")
}
+
+ def invalidAesKeyLengthError(actualLength: Int): Throwable = {
+ new SparkException(
+ s"The key length should be one of 16, 24 or 32 bytes (the actual length is $actualLength)")
Review comment:
```suggestion
s"The key length of aes_encrypt/aes_decrypt should be one of 16, 24 or 32 bytes, but got: $actualLength")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943003168
**[Test build #144240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144240/testReport)** for PR 32801 at commit [`bcb724c`](https://github.com/apache/spark/commit/bcb724c99eba1c76a51b984cb41021a3126a6105).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942996487
retest this please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942044056
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48665/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943172224
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r659262138
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +305,137 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The base implementation for AES encryption and decryption.
+ */
+abstract class AesBase(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+
+ override def dataType: DataType = BinaryType
+ override def nullable: Boolean = true
+ override def inputTypes: Seq[DataType] = Seq(BinaryType, BinaryType)
+ protected val cipherMode: Int
+
+ @transient lazy protected val cipher: Cipher = try {
+ Cipher.getInstance("AES")
+ } catch {
+ case e @ (_: NoSuchPaddingException | _: NoSuchAlgorithmException) =>
+ throw new RuntimeException(e)
+ }
+
+ protected override def nullSafeEval(input1: Any, input2: Any): Any = {
+ val input = input1.asInstanceOf[Array[Byte]]
+ val key = input2.asInstanceOf[Array[Byte]]
+ val inputLength = input.length
+ val keyLength = key.length
+ val secretKey = keyLength match {
+ case 16 | 24 | 32 => new SecretKeySpec(key, 0, keyLength, "AES")
+ case _ => null
+ }
+
+ if (secretKey == null) {
+ return null
+ }
+
+ try {
+ cipher.init(cipherMode, secretKey)
+ cipher.doFinal(input, 0, inputLength)
+ } catch {
+ case _: GeneralSecurityException =>
+ null
+ }
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
+ val inputLength = ctx.freshName("inputLength")
+ val keyLength = ctx.freshName("keyLength")
+ val secretKey = ctx.freshName("secretKey")
+ val cipher = ctx.addMutableState("javax.crypto.Cipher", "cihper", v =>
+ s"""
+ try {
+ $v = javax.crypto.Cipher.getInstance("AES");
+ } catch (javax.crypto.NoSuchPaddingException e) {
+ throw new RuntimeException(e);
+ } catch (java.security.NoSuchAlgorithmException e) {
+ throw new RuntimeException(e);
+ }""")
+
+ nullSafeCodeGen(ctx, ev, (eval1, eval2) => {
+ s"""
+ int $inputLength = $eval1.length;
+ int $keyLength = $eval2.length;
+ javax.crypto.SecretKey $secretKey = null;
+ if ($keyLength == 16 || $keyLength == 24 || $keyLength == 32) {
+ $secretKey = new javax.crypto.spec.SecretKeySpec($eval2, 0, $keyLength, "AES");
+ } else {
+ ${ev.isNull} = true;
+ }
+ if ($secretKey == null) {
+ ${ev.isNull} = true;
+ } else {
+ try {
+ $cipher.init($cipherMode, $secretKey);
+ ${ev.value} = $cipher.doFinal($eval1, 0, $inputLength);
+ } catch (java.security.GeneralSecurityException e) {
+ ${ev.isNull} = true;
+ }
+ }
+ """
+ })
+ }
+}
+
+/**
+ * A function that encrypts input using AES. Key lengths of 128, 192 or 256 bits can be used.
+ * For versions prior to JDK 8u161, 192 and 256 bits keys can be used
+ * if Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed.
+ * If either argument is NULL or the key length is not one of the permitted values,
+ * the return value is NULL.
+ */
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr, key) - Returns a encrypted value of `expr` using AES.
Review comment:
nit. `a` -> `an`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855885430
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139408/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862079173
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139831/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855710275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942549986
**[Test build #144208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144208/testReport)** for PR 32801 at commit [`29cb68f`](https://github.com/apache/spark/commit/29cb68f92ea893e910816dbf348343170562f9b5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r728616268
##########
File path: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions;
+
+import org.apache.spark.sql.errors.QueryExecutionErrors;
+
+import javax.crypto.Cipher;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.GeneralSecurityException;
+
+/**
+ * An utility class for constructing expressions.
+ */
+public class ExpressionImplUtils {
+ public static byte[] aesEncrypt(byte[] input, byte[] key) {
Review comment:
Oops. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942549986
**[Test build #144208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144208/testReport)** for PR 32801 at commit [`29cb68f`](https://github.com/apache/spark/commit/29cb68f92ea893e910816dbf348343170562f9b5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942726004
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144206/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942191312
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144186/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942993445
**[Test build #144236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144236/testReport)** for PR 32801 at commit [`bcb724c`](https://github.com/apache/spark/commit/bcb724c99eba1c76a51b984cb41021a3126a6105).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938364202
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144005/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942044056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942765691
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144208/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855710275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862186554
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139836/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-861996098
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44360/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855623854
**[Test build #139400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139400/testReport)** for PR 32801 at commit [`eb7d86f`](https://github.com/apache/spark/commit/eb7d86f1b1872618949d86fac3c049515104ed58).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855772815
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43927/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-869100942
cc @rxin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855884361
**[Test build #139405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139405/testReport)** for PR 32801 at commit [`b65326c`](https://github.com/apache/spark/commit/b65326c0896018fe592486b77f5b6232a038a597).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862047380
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44364/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942932088
**[Test build #144236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144236/testReport)** for PR 32801 at commit [`bcb724c`](https://github.com/apache/spark/commit/bcb724c99eba1c76a51b984cb41021a3126a6105).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938296716
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48482/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938256137
**[Test build #144005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144005/testReport)** for PR 32801 at commit [`82443ef`](https://github.com/apache/spark/commit/82443ef5f860b6591ace1008616cf09c253dd744).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942932088
**[Test build #144236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144236/testReport)** for PR 32801 at commit [`bcb724c`](https://github.com/apache/spark/commit/bcb724c99eba1c76a51b984cb41021a3126a6105).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942538218
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48685/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-937368209
I actually think it's maybe fine to have them as SQL function only .. since many other DBMSes have it ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r728583951
##########
File path: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions;
+
+import org.apache.spark.sql.errors.QueryExecutionErrors;
+
+import javax.crypto.Cipher;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.GeneralSecurityException;
+
+/**
+ * An utility class for constructing expressions.
+ */
+public class ExpressionImplUtils {
+ public static byte[] aesEncrypt(byte[] input, byte[] key) {
Review comment:
nit: shall we consistently use 2 spaces indentation in this java file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942293003
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144192/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855883938
**[Test build #139408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139408/testReport)** for PR 32801 at commit [`7bbbb48`](https://github.com/apache/spark/commit/7bbbb48283f8cf061967cdee7ced4fdaf380de94).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855776837
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43930/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855657149
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43922/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855682806
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43922/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
sarutak commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-938235202
@HyukjinKwon I confirmed the past PR and the mainly problematic part of the PR was the asymmetry of the implementation between encryption and decription.
https://github.com/apache/spark/pull/10527#issuecomment-186523374
https://github.com/apache/spark/pull/10527#issuecomment-186523445
This PR doesn't have the issue (the return type of encryption and decryption are `ByteType` both).
I'll resolve the conflict.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r724880094
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +305,137 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The base implementation for AES encryption and decryption.
+ */
+abstract class AesBase(left: Expression, right: Expression)
+ extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+
+ override def dataType: DataType = BinaryType
+ override def nullable: Boolean = true
+ override def inputTypes: Seq[DataType] = Seq(BinaryType, BinaryType)
+ protected val cipherMode: Int
+
+ @transient lazy protected val cipher: Cipher = try {
+ Cipher.getInstance("AES")
+ } catch {
+ case e @ (_: NoSuchPaddingException | _: NoSuchAlgorithmException) =>
+ throw new RuntimeException(e)
Review comment:
does it really help to wrap with a `RuntimeException`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-861970588
**[Test build #139831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139831/testReport)** for PR 32801 at commit [`820cfac`](https://github.com/apache/spark/commit/820cfac03bfbde18100b1092dde74ebddf2b5f1c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-862019802
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44360/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r727822957
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
##########
@@ -1826,4 +1826,9 @@ object QueryExecutionErrors {
def pivotNotAfterGroupByUnsupportedError(): Throwable = {
new UnsupportedOperationException("pivot is only supported after a groupBy")
}
+
+ def invalidAesKeyLengthError(actualLength: Int): Throwable = {
+ new SparkException(
Review comment:
`RuntimeException` seems better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942536733
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48685/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943192635
**[Test build #144240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144240/testReport)** for PR 32801 at commit [`bcb724c`](https://github.com/apache/spark/commit/bcb724c99eba1c76a51b984cb41021a3126a6105).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942090358
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48671/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32801:
URL: https://github.com/apache/spark/pull/32801#discussion_r727820746
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +306,105 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The concrete implementation for AES encryption and decryption.
+ */
+object AesImpl {
Review comment:
Can we write it in java? e.g. `class ExpressionCodegenImpl` under `org.apache.spark.sql.catalyst.expressions.codegen`, and we can add more static methods in the future when we add new expressions.
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
##########
@@ -1826,4 +1826,9 @@ object QueryExecutionErrors {
def pivotNotAfterGroupByUnsupportedError(): Throwable = {
new UnsupportedOperationException("pivot is only supported after a groupBy")
}
+
+ def invalidAesKeyLengthError(actualLength: Int): Throwable = {
+ new SparkException(
Review comment:
`RuntimeException` seems better
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
##########
@@ -1826,4 +1826,9 @@ object QueryExecutionErrors {
def pivotNotAfterGroupByUnsupportedError(): Throwable = {
new UnsupportedOperationException("pivot is only supported after a groupBy")
}
+
+ def invalidAesKeyLengthError(actualLength: Int): Throwable = {
+ new SparkException(
+ s"The key length should be one of 16, 24 or 32 bytes (the actual length is $actualLength)")
Review comment:
```suggestion
s"The key length of aes_encrypt/aes_decrypt should be one of 16, 24 or 32 bytes, but got: $actualLength")
```
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +306,105 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The concrete implementation for AES encryption and decryption.
+ */
+object AesImpl {
Review comment:
Can we write it in java? e.g. `class ExpressionImplUtils` under `org.apache.spark.sql.catalyst.expressions.codegen`, and we can add more static methods in the future when we add new expressions.
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
##########
@@ -301,3 +306,105 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
override def prettyName: String = "current_user"
final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
}
+
+/**
+ * The concrete implementation for AES encryption and decryption.
+ */
+object AesImpl {
Review comment:
Can we write it in java? e.g. `class ExpressionImplUtils` under `org.apache.spark.sql.catalyst.expressions`, and we can add more static methods in the future when we add new expressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942765691
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144208/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-941956726
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-942046204
**[Test build #144192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144192/testReport)** for PR 32801 at commit [`481791a`](https://github.com/apache/spark/commit/481791aad0e8831e0d2e67eec50f4920aca9e023).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943112915
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48720/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-941984029
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48665/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-943122829
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48720/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-861970588
**[Test build #139831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139831/testReport)** for PR 32801 at commit [`820cfac`](https://github.com/apache/spark/commit/820cfac03bfbde18100b1092dde74ebddf2b5f1c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32801:
URL: https://github.com/apache/spark/pull/32801#issuecomment-855713227
**[Test build #139405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139405/testReport)** for PR 32801 at commit [`b65326c`](https://github.com/apache/spark/commit/b65326c0896018fe592486b77f5b6232a038a597).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org