You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/06/04 19:50:46 UTC
[GitHub] [spark] izchen opened a new pull request, #36768: [SPARK-39380][SQL] Ignore comment syntax in dfs command
izchen opened a new pull request, #36768:
URL: https://github.com/apache/spark/pull/36768
### What changes were proposed in this pull request?
Ignore the comment syntax in the dfs command when Spark SQL splits semicolons.
### Why are the changes needed?
In version 3.2, Spark SQL ignores semicolons in comment syntax when splitting input lines.
This behavior throws an unexpected dfs command error:
(Related PR: SPARK-33100 SPARK-30049)
```bash
mkdir testDfsCommand
echo 'abc' > testDfsCommand/a---.txt
./spark-2.4.8-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/a---.txt;'
# abc
./spark-2.4.8-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/*;'
# abc
./spark-3.2.1-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/a---.txt;'
# cat: `testDfsCommand/a---.txt;': No such file or directory
# Command -cat testDfsCommand/a---.txt; failed with exit code = 1
./spark-3.2.1-bin-hadoop2.7/bin/spark-sql -e 'dfs -cat testDfsCommand/*;'
# cat: `testDfsCommand/*;': No such file or directory
# Command -cat testDfsCommand/*; failed with exit code = 1
```
### Does this PR introduce *any* user-facing change?
No.
Because the implementation of the dfs command always does not support comment syntax, I don't think this PR will cause compatibility issues with legacy SQL.
### How was this patch tested?
Added UT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #36768: [SPARK-39380][SQL] Ignore comment syntax in dfs command
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #36768:
URL: https://github.com/apache/spark/pull/36768#issuecomment-1280110319
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #36768: [SPARK-39380][SQL] Ignore comment syntax in dfs command
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #36768: [SPARK-39380][SQL] Ignore comment syntax in dfs command
URL: https://github.com/apache/spark/pull/36768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #36768: [SPARK-39380][SQL] Ignore comment syntax in dfs command
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on PR #36768:
URL: https://github.com/apache/spark/pull/36768#issuecomment-1146741760
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] izchen commented on pull request #36768: [SPARK-39380][SQL] Ignore comment syntax in dfs command
Posted by GitBox <gi...@apache.org>.
izchen commented on PR #36768:
URL: https://github.com/apache/spark/pull/36768#issuecomment-1178975968
@maropu @yaooqinn , could you help to review this PR? :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org