You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "dcoliversun (via GitHub)" <gi...@apache.org> on 2025/03/15 07:44:02 UTC
[PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
dcoliversun opened a new pull request, #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009
## What changes were proposed in this pull request?
This PR aims to support json_object_keys function.
(Fixes: \#9008)
## How was this patch tested?
unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2767922724
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2767923090
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030371408
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +326,64 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function valid json format") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]""")
Review Comment:
nit: these two tests can be merged into a single test to verify valid, invalid, null, and encoding cases (e.g., Chinese).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030378714
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +326,64 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function valid json format") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]""")
+ )
+ .toDF("txt")
+ .write
+ .parquet(path.getCanonicalPath)
+
+ spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("tbl")
+
+ runQueryAndCompare("select txt, json_object_keys(txt) from tbl") {
+ checkSparkOperatorMatch[ProjectExecTransformer]
+ }
+
+ runQueryAndCompare(
+ "SELECT json_object_keys(null) " +
+ "from datatab limit 1;") {
+ checkGlutenOperatorMatch[ProjectExecTransformer]
+ }
+ }
+ }
+
+ test("json_object_keys function invalid json format") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ ("""{[1,2]}"""),
+ ("""{"key": 45, "random_string"}""")
+ )
+ .toDF("txt")
+ .write
+ .parquet(path.getCanonicalPath)
+
+ spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("tbl")
+
+ runQueryAndCompare("select txt, json_object_keys(txt) from tbl") {
+ checkSparkOperatorMatch[ProjectExecTransformer]
+ }
+ }
+ }
+
+ test("json_object_keys function no argument") {
+ intercept[AnalysisException] {
+ sql(
+ "SELECT json_object_keys()" +
+ "FROM datatab LIMIT 1");
Review Comment:
This test looks unnecessary. It just verifies Spark's analysis code. Please remove it if no reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2726313647
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2783357991
@PHILO-HE @jackylee-ch It's ready to review, and test fails with oom
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2764882171
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2765574873
@jackylee-ch I know why. I only check [JsonFunctionsValidateSuite.scala](https://github.com/apache/incubator-gluten/pull/9009/files#diff-e03fbee12c3c58e7e8aaccdac97670cf7cca7e85ad338af246b6b5009da3171d) added in this PR and ignore JsonFunctionsSuite in spark.
I will check it again. Thanks for your help :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2763258961
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2765749728
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2784993143
@PHILO-HE Any more questions about this pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2726313525
https://github.com/apache/incubator-gluten/issues/9008
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030486665
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +326,64 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function valid json format") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]""")
+ )
+ .toDF("txt")
+ .write
+ .parquet(path.getCanonicalPath)
+
+ spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("tbl")
+
+ runQueryAndCompare("select txt, json_object_keys(txt) from tbl") {
+ checkSparkOperatorMatch[ProjectExecTransformer]
+ }
+
+ runQueryAndCompare(
+ "SELECT json_object_keys(null) " +
+ "from datatab limit 1;") {
+ checkGlutenOperatorMatch[ProjectExecTransformer]
+ }
+ }
+ }
+
+ test("json_object_keys function invalid json format") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ ("""{[1,2]}"""),
+ ("""{"key": 45, "random_string"}""")
+ )
+ .toDF("txt")
+ .write
+ .parquet(path.getCanonicalPath)
+
+ spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("tbl")
+
+ runQueryAndCompare("select txt, json_object_keys(txt) from tbl") {
+ checkSparkOperatorMatch[ProjectExecTransformer]
+ }
+ }
+ }
+
+ test("json_object_keys function no argument") {
+ intercept[AnalysisException] {
+ sql(
+ "SELECT json_object_keys()" +
+ "FROM datatab LIMIT 1");
Review Comment:
Sure, removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030486835
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +326,64 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function valid json format") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]""")
Review Comment:
Sure, merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2781797068
@PHILO-HE @jackylee-ch I believe it's ready to review, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE merged PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2765242341
@PHILO-HE @jackylee-ch I believe it's ready to review. Relative tests pass but other unrelated tests failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2763342542
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030845868
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +325,39 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]"""),
+ ("""{[1,2]}"""),
+ ("""{"key": 45, "random_string"}""")
+ )
+ .toDF("txt")
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2782677947
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2726357332
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2765299178
@jackylee-ch It is not updated.(And I want to fix it with other PR). I have checked logs of all tests, which show JsonFunctionsValidateSuite passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2782074285
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030833984
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +325,39 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]"""),
+ ("""{[1,2]}"""),
+ ("""{"key": 45, "random_string"}""")
+ )
+ .toDF("txt")
+ .write
+ .parquet(path.getCanonicalPath)
+
+ spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("tbl")
+
+ runQueryAndCompare("select txt, json_object_keys(txt) from tbl") {
+ checkSparkOperatorMatch[ProjectExecTransformer]
+ }
+
+ runQueryAndCompare(
+ "SELECT json_object_keys(null) " +
Review Comment:
We need read null from relation, otherwise it won't offload to native side.
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +325,39 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]"""),
+ ("""{[1,2]}"""),
+ ("""{"key": 45, "random_string"}""")
+ )
+ .toDF("txt")
Review Comment:
This can be changed to parquet and write null value to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2781174565
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on code in PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#discussion_r2030844583
##########
backends-velox/src/test/scala/org/apache/gluten/execution/JsonFunctionsValidateSuite.scala:
##########
@@ -325,4 +325,39 @@ class JsonFunctionsValidateSuite extends FunctionsValidateSuite {
}
}
}
+
+ test("json_object_keys function") {
+ withTempPath {
+ path =>
+ Seq[(String)](
+ (""""""),
+ ("""200"""),
+ ("""{}"""),
+ ("""{"key": 1}"""),
+ ("""{"key": "value", "key2": 2}"""),
+ ("""{"arrayKey": [1, 2, 3]}"""),
+ ("""{"key":[1,2,3,{"key":"value"},[1,2,3]]}"""),
+ ("""{"f1":"abc","f2":{"f3":"a", "f4":"b"}}"""),
+ ("""{"k1": [1, 2, {"key": 5}], "k2": {"key2": [1, 2]}}"""),
+ ("""[1, 2, 3]"""),
+ ("""{[1,2]}"""),
+ ("""{"key": 45, "random_string"}""")
+ )
+ .toDF("txt")
+ .write
+ .parquet(path.getCanonicalPath)
+
+ spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("tbl")
+
+ runQueryAndCompare("select txt, json_object_keys(txt) from tbl") {
+ checkSparkOperatorMatch[ProjectExecTransformer]
+ }
+
+ runQueryAndCompare(
+ "SELECT json_object_keys(null) " +
Review Comment:
Fine, I see other test with `select spark_function(null) from tbl` also. I will submit other PR to fix, this case is ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2782621221
@PHILO-HE @jackylee-ch The comments are addressed. Some tests failed and no one is relative to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2765256734
https://github.com/apache/incubator-gluten/pull/9009/checks?check_run_id=39674902439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9008][VL] Support json_object_keys function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9009:
URL: https://github.com/apache/incubator-gluten/pull/9009#issuecomment-2767918456
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org