You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by kiszk <gi...@git.apache.org> on 2015/12/27 09:30:05 UTC
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
GitHub user kiszk opened a pull request:
https://github.com/apache/spark/pull/10488
[SPARK-12530][Build] Fix build break at Spark-Master-Maven-Snapshots from #1293
Compilation error caused due to string concatenations that are not a constant
Use raw string literal to avoid string concatenations
https://amplab.cs.berkeley.edu/jenkins/view/Spark-Packaging/job/Spark-Master-Maven-Snapshots/1293/
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kiszk/spark SPARK-12530
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10488.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10488
----
commit 9658c3b6b66034a515a3e5d3cfd3d6cc9c4a3305
Author: Kazuaki Ishizaki <is...@jp.ibm.com>
Date: 2015-12-27T08:24:38Z
fix compilation error
use raw string literal to avoid string concatenations
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48499260
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
@kiszk Will
```
extended = "> SELECT _FUNC_('Spark', 0);\n " +
"'529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b'")
```
cause the same issue?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167853136
I see. I will create another JIRA entry to update other usages.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48534869
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
In another way, I am trying to reproduce this compilation error in a standalone test case. However, I cannot reproduce this failure (i.e. compilations are successfully finished).
```
$ cat ExpressionDescription.java
import java.lang.annotation.*;
@Retention(RetentionPolicy.RUNTIME)
public @interface ExpressionDescription {
String usage() default "_FUNC_ is undocumented";
String extended() default "No example for _FUNC_.";
}
$ cat test.scala
@ExpressionDescription(
usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
"input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
"to 256",
extended = "> SELECT _FUNC_('Spark', 0);\n " +
"'529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b'")
class test {
def func() : Unit = {
val str = "abc" + "123" + "XYZ"
}
}
$ javac ExpressionDescription.java
$ ~/scala-2.11.7/bin/scalac test.scala
$
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167630559
I am going to merge this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48500579
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
@yhuai that is the weird thing. One concatenation is fine, more than one isn't. Seems like a bug in scala :s...
I think it is better to be consistent and to only allow raw strings in case of multi-line ```usage/extended``` fields.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48472393
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
Yes, this keeps within 100 characters at a line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167401572
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167506758
More people are having this problem. Lets use raw strings then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48499035
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
@kiszk Thank you for checking. How about we still use multi-line string?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167418258
There a quite a few builds passing which are using string concatenation in combination with ```ExpressionDescription```:
https://github.com/apache/spark/pull/10489
https://github.com/apache/spark/pull/10418
https://github.com/apache/spark/pull/10402
Maybe something else is causing this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48498744
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
I confirmed ``// scalastyle:off`` and ``// scalastyle:on`` if we use up to one string concatenation for `` usage = ``.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48475897
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
Does that work under 2.11?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167401573
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48348/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167424111
@hvanhovell, a set of following two conditions causes this compilation failure:
1. Use more than one string concatenations
2. Use scala 2.11 compiler (open question is that [this failure](https://amplab.cs.berkeley.edu/jenkins/view/Spark-Packaging/job/Spark-Master-Maven-Snapshots/1293/consoleFull) occurs by using both scala 2.10 and 2.11 compilers on Jenkins
I can reproduce this compilation error by using the following command:
````
$ java -version
openjdk version "1.8.0_65"
OpenJDK Runtime Environment (build 1.8.0_65-b17)
OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)
$ build/mvn -Dscala-2.11 -Pyarn -Phadoop-2.4 -DskipTests clean package install -pl sql/catalyst
````
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48500799
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
@yhuai , the above line did not cause the same issue. This is because its length is less than 100 and it has only one concatenation.
I checked by inserting ``// scalastyle:off`` before the annotation and ``// scalastyle:on`` after the annotation as follows:
```
// scalastyle:off
@ExpressionDescription(
usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
"input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256",
extended = "> SELECT _FUNC_('Spark', 0);\n " +
"'529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b'")
// scalastyle:on
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/10488
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by vectorijk <gi...@git.apache.org>.
Github user vectorijk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48475242
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
Style below make newline display more reasonable.
```scala
s"""first line
| second line
""".stripMargin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167540178
@yhuai can you take a look at this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48503757
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
@kiszk I see. How about we remove concatenation from both `usage` and `extended`? We can just use multi-line string.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48514250
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
If we use the multi-line string, we need to be careful when adding `\n`. This does not work. I already hit this issue when I do the fix in another PR: https://github.com/apache/spark/pull/10418
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167848836
@yhuai, do you mean that I would update all of the string concatenation in @ExpressionDescription by using multi-line string literals rather than only the original one?
If so, I will do this update.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167418794
Thanks for letting me know them. I will check them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48479722
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
I have just tried this; it doesn't work unfortunately.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167401548
**[Test build #48348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48348/consoleFull)** for PR 10488 at commit [`9658c3b`](https://github.com/apache/spark/commit/9658c3b6b66034a515a3e5d3cfd3d6cc9c4a3305).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48534614
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
@yhuai, it sounds good to use multi-line string for ``usage`` and ``extended``. How about this policy?
1. First, we use raw string literal to remove concatenations.
2. If we still want to use a line with more than 100 characters, ``// scalastyle:off`` and ``// scalastyle:on`` can be used.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167834296
Can you update the pr? Once it passes jenkins, I will merge it. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48495327
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
I am wondering if we can just turn off the style check for this and have a long one-line string constant. You can use `// scalastyle:off` and `// scalastyle:on`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/10488#discussion_r48470392
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala ---
@@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
- usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
- "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
- "to 256",
+ usage =
+ """_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
+ SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
+ ,
--- End diff --
Nit style. I guess this was needed to stay within 100 charaters?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167396615
**[Test build #48348 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48348/consoleFull)** for PR 10488 at commit [`9658c3b`](https://github.com/apache/spark/commit/9658c3b6b66034a515a3e5d3cfd3d6cc9c4a3305).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167834026
@kiszk Thank you for the investigation. Yeah, let's use multi-line string literals. If we have to have a line with more than 100 characters, let's use `// scalastyle:off line.size.limit` and `// scalastyle:on line.size.limit` to just bypass the line number requirement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/10488#issuecomment-167849606
Let me merge this one first to fix the spark master maven snapshot. Then, can you create another jira to update other places?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org