You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/05 07:43:08 UTC
[GitHub] [spark] MaxGekk opened a new pull request #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
MaxGekk opened a new pull request #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804
### What changes were proposed in this pull request?
In the PR, I propose:
1. To replace matching by `Literal` in `ExprUtils.evalSchemaExpr()` to checking foldable property of the `schema` expression.
2. To replace matching by `Literal` in `ExprUtils.evalTypeExpr()` to checking foldable property of the `schema` expression.
3. To change checking of the input parameter in the `SchemaOfCsv` expression, and allow foldable `child` expression.
4. To change checking of the input parameter in the `SchemaOfJson` expression, and allow foldable `child` expression.
Closes #27771
Closes #27774
Closes #27777
Closes #27797
### Why are the changes needed?
This should improve Spark SQL UX for `from_csv`/`from_json`. Currently, Spark expects only literals:
```sql
spark-sql> select from_csv('1,Moscow', replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', ''));
Error in query: Schema should be specified in DDL format as a string literal or output of the schema_of_csv function instead of replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', '');; line 1 pos 7
spark-sql> select from_json('{"id":1, "city":"Moscow"}', replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', ''));
Error in query: Schema should be specified in DDL format as a string literal or output of the schema_of_json function instead of replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', '');; line 1 pos 7
```
and only string literals are acceptable as CSV examples by `schema_of_csv`/`schema_of_json`:
```sql
spark-sql> select schema_of_csv(concat_ws(',', 0.1, 1));
Error in query: cannot resolve 'schema_of_csv(concat_ws(',', CAST(0.1BD AS STRING), CAST(1 AS STRING)))' due to data type mismatch: The input csv should be a string literal and not null; however, got concat_ws(',', CAST(0.1BD AS STRING), CAST(1 AS STRING)).; line 1 pos 7;
'Project [unresolvedalias(schema_of_csv(concat_ws(,, cast(0.1 as string), cast(1 as string))), None)]
+- OneRowRelation
spark-sql> select schema_of_json(regexp_replace('{"item_id": 1, "item_price": 0.1}', 'item_', ''));
Error in query: cannot resolve 'schema_of_json(regexp_replace('{"item_id": 1, "item_price": 0.1}', 'item_', ''))' due to data type mismatch: The input json should be a string literal and not null; however, got regexp_replace('{"item_id": 1, "item_price": 0.1}', 'item_', '').; line 1 pos 7;
'Project [unresolvedalias(schema_of_json(regexp_replace({"item_id": 1, "item_price": 0.1}, item_, )), None)]
+- OneRowRelation
```
### Does this PR introduce any user-facing change?
Yes, after the changes users can pass any foldable string expression as the `schema` parameter to `from_csv()/from_json()`. For the example above:
```sql
spark-sql> select from_csv('1,Moscow', replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', ''));
{"id":1,"city":"Moscow"}
spark-sql> select from_json('{"id":1, "city":"Moscow"}', replace('dpt_org_id INT, dpt_org_city STRING', 'dpt_org_', ''));
{"id":1,"city":"Moscow"}
```
After change the `schema_of_csv`/`schema_of_json` functions accept foldable expressions, for example:
```sql
spark-sql> select schema_of_csv(concat_ws(',', 0.1, 1));
struct<_c0:double,_c1:int>
spark-sql> select schema_of_json(regexp_replace('{"item_id": 1, "item_price": 0.1}', 'item_', ''));
struct<id:bigint,price:double>
```
### How was this patch tested?
Added new test to `CsvFunctionsSuite` and to `JsonFunctionsSuite`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595084189
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595348333
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24145/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595348319
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595347610
**[Test build #119408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119408/testReport)** for PR 27804 at commit [`874e17b`](https://github.com/apache/spark/commit/874e17b70b8af45ba7968bbf4b979f3623b399f0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595100451
jenkins, retest this, please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595104320
**[Test build #119375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119375/testReport)** for PR 27804 at commit [`cd93715`](https://github.com/apache/spark/commit/cd93715861f58035e61fc240ff5eca590757b5b9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595084189
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595412393
**[Test build #119415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119415/testReport)** for PR 27804 at commit [`87c2cf8`](https://github.com/apache/spark/commit/87c2cf8f5d7bf1167ea32c3c39e84e075278f984).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595520378
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595203449
I'm fine as well, though I don't think it has a large impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595520381
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119415/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595101225
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24112/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595404854
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595519803
**[Test build #119415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119415/testReport)** for PR 27804 at commit [`87c2cf8`](https://github.com/apache/spark/commit/87c2cf8f5d7bf1167ea32c3c39e84e075278f984).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595084201
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119371/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595076195
**[Test build #119371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119371/testReport)** for PR 27804 at commit [`cd93715`](https://github.com/apache/spark/commit/cd93715861f58035e61fc240ff5eca590757b5b9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595104320
**[Test build #119375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119375/testReport)** for PR 27804 at commit [`cd93715`](https://github.com/apache/spark/commit/cd93715861f58035e61fc240ff5eca590757b5b9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595413002
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24154/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595076674
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595594738
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595074873
@HyukjinKwon @cloud-fan Please, review the PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595101225
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24112/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595520381
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119415/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595115776
As mentioned in other PRs, I am okay. I don't feel strongly. Let me know if you guys have preferences @cloud-fan and @dongjoon-hyun.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595412393
**[Test build #119415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119415/testReport)** for PR 27804 at commit [`87c2cf8`](https://github.com/apache/spark/commit/87c2cf8f5d7bf1167ea32c3c39e84e075278f984).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595228254
**[Test build #119375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119375/testReport)** for PR 27804 at commit [`cd93715`](https://github.com/apache/spark/commit/cd93715861f58035e61fc240ff5eca590757b5b9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#discussion_r388260598
##########
File path: sql/core/src/test/resources/sql-tests/results/csv-functions.sql.out
##########
@@ -24,7 +24,7 @@ select from_csv('1', 1)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-Schema should be specified in DDL format as a string literal or output of the schema_of_csv function instead of 1;; line 1 pos 7
+The expression '1' must be evaluated to a valid string.;; line 1 pos 7
Review comment:
`valid schema string`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595101215
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595076684
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24108/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595404859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119408/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595084140
**[Test build #119371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119371/testReport)** for PR 27804 at commit [`cd93715`](https://github.com/apache/spark/commit/cd93715861f58035e61fc240ff5eca590757b5b9).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595404854
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595413002
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24154/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595520378
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595404859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119408/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#discussion_r388260818
##########
File path: sql/core/src/test/resources/sql-tests/results/csv-functions.sql.out
##########
@@ -24,7 +24,7 @@ select from_csv('1', 1)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-Schema should be specified in DDL format as a string literal or output of the schema_of_csv function instead of 1;; line 1 pos 7
+The expression '1' must be evaluated to a valid string.;; line 1 pos 7
Review comment:
and as an error message, seems better to have `The expression '1' is not a valid schema string.`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595347610
**[Test build #119408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119408/testReport)** for PR 27804 at commit [`874e17b`](https://github.com/apache/spark/commit/874e17b70b8af45ba7968bbf4b979f3623b399f0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595412993
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595229412
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595076674
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595076684
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24108/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595084201
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119371/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595412993
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595076195
**[Test build #119371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119371/testReport)** for PR 27804 at commit [`cd93715`](https://github.com/apache/spark/commit/cd93715861f58035e61fc240ff5eca590757b5b9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595348319
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595404681
**[Test build #119408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119408/testReport)** for PR 27804 at commit [`874e17b`](https://github.com/apache/spark/commit/874e17b70b8af45ba7968bbf4b979f3623b399f0).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595229419
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119375/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595229412
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595348333
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24145/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#discussion_r388441714
##########
File path: sql/core/src/test/resources/sql-tests/results/csv-functions.sql.out
##########
@@ -24,7 +24,7 @@ select from_csv('1', 1)
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-Schema should be specified in DDL format as a string literal or output of the schema_of_csv function instead of 1;; line 1 pos 7
+The expression '1' must be evaluated to a valid string.;; line 1 pos 7
Review comment:
replaced
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595101215
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27804:
[SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable
args by `from_csv/json` and `schema_of_csv/json`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27804: [SPARK-31020][SPARK-31023][SPARK-31025][SPARK-31044][SQL] Support foldable args by `from_csv/json` and `schema_of_csv/json`
URL: https://github.com/apache/spark/pull/27804#issuecomment-595229419
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119375/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org