You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2022/11/03 13:32:53 UTC
[spark] branch master updated: [SPARK-40996][BUILD] Upgrade `sbt-checkstyle-plugin` to 4.0.0 to resolve `dev/sbt-checkstyle` run failed with sbt 1.7.3
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 67d0dc8f74d [SPARK-40996][BUILD] Upgrade `sbt-checkstyle-plugin` to 4.0.0 to resolve `dev/sbt-checkstyle` run failed with sbt 1.7.3
67d0dc8f74d is described below
commit 67d0dc8f74d78678446f190145a34f71e60efa99
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Thu Nov 3 08:32:41 2022 -0500
[SPARK-40996][BUILD] Upgrade `sbt-checkstyle-plugin` to 4.0.0 to resolve `dev/sbt-checkstyle` run failed with sbt 1.7.3
### What changes were proposed in this pull request?
This pr aims upgrade `sbt-checkstyle-plugin` to 4.0.0 to resolve `dev/sbt-checkstyle` run failed with sbt 1.7.3, the new version will check the generated source code, so some new suppression rules have been added to `dev/checkstyle-suppressions.xml`
### Why are the changes needed?
https://github.com/apache/spark/pull/38476 revert sbt 1.7.3 upgrade due to run `dev/sbt-checkstyle` failed:
```
[error] org.xml.sax.SAXParseException; lineNumber: 18; columnNumber: 10; DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
[error] at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
[error] at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
[error] at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
[error] at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
[error] at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1473)
[error] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:914)
[error] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
[error] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
[error] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
[error] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
[error] at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
[error] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
[error] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
[error] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
[error] at scala.xml.factory.XMLLoader.parse(XMLLoader.scala:73)
[error] at scala.xml.factory.XMLLoader.loadXML(XMLLoader.scala:54)
[error] at scala.xml.factory.XMLLoader.loadXML$(XMLLoader.scala:53)
[error] at scala.xml.XML$.loadXML(XML.scala:62)
[error] at scala.xml.factory.XMLLoader.loadString(XMLLoader.scala:92)
[error] at scala.xml.factory.XMLLoader.loadString$(XMLLoader.scala:92)
[error] at scala.xml.XML$.loadString(XML.scala:62)
[error] at com.etsy.sbt.checkstyle.Checkstyle$.checkstyle(Checkstyle.scala:35)
[error] at com.etsy.sbt.checkstyle.CheckstylePlugin$autoImport$.$anonfun$checkstyleTask$1(CheckstylePlugin.scala:36)
[error] at com.etsy.sbt.checkstyle.CheckstylePlugin$autoImport$.$anonfun$checkstyleTask$1$adapted(CheckstylePlugin.scala:34)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] at sbt.Execute.work(Execute.scala:291)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
```
This pr upgrade `sbt-checkstyle-plugin` to 4.0.0 to resolve the above issue, the `https://github.com/stringbean/sbt-checkstyle-plugin` was forked from etsy/sbt-checkstyle-plugin in 2022 after it became unmaintained, the release notes as follows:
- https://github.com/stringbean/sbt-checkstyle-plugin/releases/tag/3.2.0
- https://github.com/stringbean/sbt-checkstyle-plugin/releases/tag/3.3.0
- https://github.com/stringbean/sbt-checkstyle-plugin/releases/tag/v4.0.0
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Manual test `dev/sbt-checkstyle` with sbt 1.7.3 and this pr: `Checkstyle checks passed.`
Closes #38481 from LuciferYang/173-pass-checkstyle.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Sean Owen <sr...@gmail.com>
---
dev/checkstyle-suppressions.xml | 8 ++++++++
project/SparkBuild.scala | 2 +-
project/plugins.sbt | 2 +-
3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/dev/checkstyle-suppressions.xml b/dev/checkstyle-suppressions.xml
index d48018caf87..ce569464231 100644
--- a/dev/checkstyle-suppressions.xml
+++ b/dev/checkstyle-suppressions.xml
@@ -34,6 +34,14 @@
files="connector/spark-ganglia-lgpl/src/main/java/com/codahale/metrics/ganglia/GangliaReporter.java"/>
<suppress checks=".*"
files="sql/core/src/main/java/org/apache/spark/sql/api/java/*"/>
+ <suppress checks=".*"
+ files="antlr4/org/apache/spark/sql/catalyst/parser/*"/>
+ <suppress checks=".*"
+ files="org/apache/spark/connect/proto/*"/>
+ <suppress checks=".*"
+ files="test/gen-java/org/apache/spark/sql/execution/datasources/parquet/test/avro/*"/>
+ <suppress checks=".*"
+ files="generated-test-sources/org/apache/spark/sql/protobuf/*"/>
<suppress checks="LineLength"
files="src/test/java/org/apache/spark/sql/hive/test/Complex.java"/>
<suppress checks="LineLength"
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 03baa77090e..f61753f7b1d 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -1269,7 +1269,7 @@ object Unidoc {
object Checkstyle {
lazy val settings = Seq(
- checkstyleSeverityLevel := Some(CheckstyleSeverityLevel.Error),
+ checkstyleSeverityLevel := CheckstyleSeverityLevel.Error,
(Compile / checkstyle / javaSource) := baseDirectory.value / "src/main/java",
(Test / checkstyle / javaSource) := baseDirectory.value / "src/test/java",
checkstyleConfigLocation := CheckstyleConfigLocation.File("dev/checkstyle.xml"),
diff --git a/project/plugins.sbt b/project/plugins.sbt
index 32ee76831f7..c0c418bd521 100644
--- a/project/plugins.sbt
+++ b/project/plugins.sbt
@@ -15,7 +15,7 @@
* limitations under the License.
*/
-addSbtPlugin("com.etsy" % "sbt-checkstyle-plugin" % "3.1.1")
+addSbtPlugin("software.purpledragon" % "sbt-checkstyle-plugin" % "4.0.0")
// sbt-checkstyle-plugin uses an old version of checkstyle. Match it to Maven's.
// If you are changing the dependency setting for checkstyle plugin,
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org