You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Koert Kuipers <ko...@tresata.com> on 2014/10/23 20:03:24 UTC
scalastyle annoys me a little bit
100 max width seems very restrictive to me.
even the most restrictive environment i have for development (ssh with
emacs) i get a lot more characters to work with than that.
personally i find the code harder to read, not easier. like i kept
wondering why there are weird newlines in the
middle of constructors and such, only to realise later it was because of
the 100 character limit.
also, i find "mvn package" erroring out because of style errors somewhat
excessive. i understand that a pull request needs to conform to "the style"
before being accepted, but this means i cant even run tests on code that
does not conform to the style guide, which is a bit silly.
i keep going out for coffee while package and tests run, only to come back
for an annoying error that my line is 101 characters and therefore nothing
ran.
is there some maven switch to disable the style checks?
best! koert
Re: scalastyle annoys me a little bit
Posted by Koert Kuipers <ko...@tresata.com>.
oh i found some stuff about tests and how to continue them, gonna try that
now (-fae switch). should have googled before asking...
On Fri, Oct 24, 2014 at 3:59 PM, Koert Kuipers <ko...@tresata.com> wrote:
> thanks ted.
>
> apologies for complaining about maven here again, but this is the first
> time i seriously use it for development, and i am completely unfamiliar
> with it.
>
> a few more issues:
>
> "mvn clean package -DskipTests" takes about 30 mins for me. thats painful
> since its needed for the tests. does anyone know any tricks to speed it up?
> (besides getting a better laptop). does zinc help?
>
> does scalastyle overwrite files? after i do "mvn package" emacs is
> completely confused and for every file it says "it has been edited" and i
> need to re-open it. not helpful for a development cycle. i think i am
> simply going to edit the pom and remove scalacheck for development.
>
> mvn test runs through the projects until one fails. then it skips the
> rest! since its very likely that i get a failure in some subproject, this
> means its nearly impossible to do a general test run and get a good sense
> of the status of the project. for example:
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project Parent POM .......................... SUCCESS [2.199s]
> [INFO] Spark Project Core ................................ SUCCESS
> [39:43.028s]
> [INFO] Spark Project Bagel ............................... SUCCESS
> [42.569s]
> [INFO] Spark Project GraphX .............................. SUCCESS
> [3:22.104s]
> [INFO] Spark Project Streaming ........................... SUCCESS
> [7:12.592s]
> [INFO] Spark Project ML Library .......................... SUCCESS
> [10:32.682s]
> [INFO] Spark Project Tools ............................... SUCCESS
> [17.070s]
> [INFO] Spark Project Catalyst ............................ SUCCESS
> [3:03.470s]
> [INFO] Spark Project SQL ................................. SUCCESS
> [5:23.993s]
> [INFO] Spark Project Hive ................................ FAILURE
> [2:08.387s]
> [INFO] Spark Project REPL ................................ SKIPPED
> [INFO] Spark Project Assembly ............................ SKIPPED
> [INFO] Spark Project External Twitter .................... SKIPPED
> [INFO] Spark Project External Kafka ...................... SKIPPED
> [INFO] Spark Project External Flume Sink ................. SKIPPED
> [INFO] Spark Project External Flume ...................... SKIPPED
> [INFO] Spark Project External ZeroMQ ..................... SKIPPED
> [INFO] Spark Project External MQTT ....................... SKIPPED
> [INFO] Spark Project Examples ............................ SKIPPED
>
> in this case i dont care about Hive, but i would have liked to see REPL
> run, and Kafka.
>
>
>
>
> On Thu, Oct 23, 2014 at 4:44 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Created SPARK-4066 and attached patch there.
>>
>> On Thu, Oct 23, 2014 at 1:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
>>
>>> great thanks i will do that
>>>
>>> On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yu...@gmail.com> wrote:
>>>
>>>> Koert:
>>>> If you have time, you can try this diff - with which you would be able
>>>> to specify the following on the command line:
>>>> -Dscalastyle.failonviolation=false
>>>>
>>>> diff --git a/pom.xml b/pom.xml
>>>> index 687cc63..108585e 100644
>>>> --- a/pom.xml
>>>> +++ b/pom.xml
>>>> @@ -123,6 +123,7 @@
>>>> <log4j.version>1.2.17</log4j.version>
>>>> <hadoop.version>1.0.4</hadoop.version>
>>>> <protobuf.version>2.4.1</protobuf.version>
>>>> + <scalastyle.failonviolation>true</scalastyle.failonviolation>
>>>> <yarn.version>${hadoop.version}</yarn.version>
>>>> <hbase.version>0.94.6</hbase.version>
>>>> <flume.version>1.4.0</flume.version>
>>>> @@ -1071,7 +1072,7 @@
>>>> <version>0.4.0</version>
>>>> <configuration>
>>>> <verbose>false</verbose>
>>>> - <failOnViolation>true</failOnViolation>
>>>> +
>>>> <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
>>>>
>>>> <includeTestSourceDirectory>false</includeTestSourceDirectory>
>>>> <failOnWarning>false</failOnWarning>
>>>> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
>>>>
>>>>
>>>>
>>>> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com>
>>>> wrote:
>>>>
>>>>> Hey Ted,
>>>>> i tried:
>>>>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>>>>>
>>>>> no luck, still get
>>>>> [ERROR] Failed to execute goal
>>>>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
>>>>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
>>>>> violation(s). -> [Help 1]
>>>>>
>>>>>
>>>>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>>
>>>>>> Koert:
>>>>>> Have you tried adding the following on your commandline ?
>>>>>>
>>>>>> -Dscalastyle.failOnViolation=false
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pwendell@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hey Koert,
>>>>>>>
>>>>>>> I think disabling the style checks in maven package could be a good
>>>>>>> idea for the reason you point out. I was sort of mixed on that when
>>>>>>> it
>>>>>>> was proposed for this exact reason. It's just annoying to developers.
>>>>>>>
>>>>>>> In terms of changing the global limit, this is more religion than
>>>>>>> anything else, but there are other cases where the current limit is
>>>>>>> useful (e.g. if you have many windows open in a large screen).
>>>>>>>
>>>>>>> - Patrick
>>>>>>>
>>>>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>>>>>> wrote:
>>>>>>> > 100 max width seems very restrictive to me.
>>>>>>> >
>>>>>>> > even the most restrictive environment i have for development (ssh
>>>>>>> with
>>>>>>> > emacs) i get a lot more characters to work with than that.
>>>>>>> >
>>>>>>> > personally i find the code harder to read, not easier. like i kept
>>>>>>> > wondering why there are weird newlines in the
>>>>>>> > middle of constructors and such, only to realise later it was
>>>>>>> because of
>>>>>>> > the 100 character limit.
>>>>>>> >
>>>>>>> > also, i find "mvn package" erroring out because of style errors
>>>>>>> somewhat
>>>>>>> > excessive. i understand that a pull request needs to conform to
>>>>>>> "the style"
>>>>>>> > before being accepted, but this means i cant even run tests on
>>>>>>> code that
>>>>>>> > does not conform to the style guide, which is a bit silly.
>>>>>>> >
>>>>>>> > i keep going out for coffee while package and tests run, only to
>>>>>>> come back
>>>>>>> > for an annoying error that my line is 101 characters and therefore
>>>>>>> nothing
>>>>>>> > ran.
>>>>>>> >
>>>>>>> > is there some maven switch to disable the style checks?
>>>>>>> >
>>>>>>> > best! koert
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Koert Kuipers <ko...@tresata.com>.
thanks everyone, very helpful
On Fri, Oct 24, 2014 at 4:22 PM, Stephen Boesch <ja...@gmail.com> wrote:
> Sean Owen beat me to (strongly) recommending running zinc server. Using
> the -pl option is great too - but be careful to only use it when your work
> is restricted to the modules in the (comma separated) list you provide to
> -pl. Also before using -pl you should do a mvn compile package install
> on all modules. Use the -pl after those steps are done - and then it is
> very effective.
>
> 2014-10-24 13:08 GMT-07:00 Sean Owen <so...@cloudera.com>:
>
>> On Fri, Oct 24, 2014 at 8:59 PM, Koert Kuipers <ko...@tresata.com> wrote:
>> > "mvn clean package -DskipTests" takes about 30 mins for me. thats
>> painful
>> > since its needed for the tests. does anyone know any tricks to speed it
>> up?
>> > (besides getting a better laptop). does zinc help?
>>
>> Zinc helps by about 50-100%. Worthwhile for sure. brew install zinc
>> and zinc -start
>>
>> > mvn test runs through the projects until one fails. then it skips the
>> rest!
>> > since its very likely that i get a failure in some subproject, this
>> means
>> > its nearly impossible to do a general test run and get a good sense of
>> the
>> > status of the project. for example:
>>
>> You can mvn test -pl [module] to test just one module.
>> It will also indicate to you that after a failure you can mvn test -rf
>> :[module] to continue where it left off -- you can use this to resume
>> at the next module.
>>
>> Or try "-Dscalatest.testFailureIgnore=true" if the mvn flags
>> themselves don't work, for continuing after a test failure.
>>
>> > [INFO]
>> > ------------------------------------------------------------------------
>> > [INFO] Reactor Summary:
>> > [INFO]
>> > [INFO] Spark Project Parent POM .......................... SUCCESS
>> [2.199s]
>> > [INFO] Spark Project Core ................................ SUCCESS
>> > [39:43.028s]
>> > [INFO] Spark Project Bagel ............................... SUCCESS
>> [42.569s]
>> > [INFO] Spark Project GraphX .............................. SUCCESS
>> > [3:22.104s]
>> > [INFO] Spark Project Streaming ........................... SUCCESS
>> > [7:12.592s]
>> > [INFO] Spark Project ML Library .......................... SUCCESS
>> > [10:32.682s]
>> > [INFO] Spark Project Tools ............................... SUCCESS
>> [17.070s]
>> > [INFO] Spark Project Catalyst ............................ SUCCESS
>> > [3:03.470s]
>> > [INFO] Spark Project SQL ................................. SUCCESS
>> > [5:23.993s]
>> > [INFO] Spark Project Hive ................................ FAILURE
>> > [2:08.387s]
>> > [INFO] Spark Project REPL ................................ SKIPPED
>> > [INFO] Spark Project Assembly ............................ SKIPPED
>> > [INFO] Spark Project External Twitter .................... SKIPPED
>> > [INFO] Spark Project External Kafka ...................... SKIPPED
>> > [INFO] Spark Project External Flume Sink ................. SKIPPED
>> > [INFO] Spark Project External Flume ...................... SKIPPED
>> > [INFO] Spark Project External ZeroMQ ..................... SKIPPED
>> > [INFO] Spark Project External MQTT ....................... SKIPPED
>> > [INFO] Spark Project Examples ............................ SKIPPED
>> >
>> > in this case i dont care about Hive, but i would have liked to see REPL
>> > run, and Kafka.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Stephen Boesch <ja...@gmail.com>.
Sean Owen beat me to (strongly) recommending running zinc server. Using
the -pl option is great too - but be careful to only use it when your work
is restricted to the modules in the (comma separated) list you provide to
-pl. Also before using -pl you should do a mvn compile package install
on all modules. Use the -pl after those steps are done - and then it is
very effective.
2014-10-24 13:08 GMT-07:00 Sean Owen <so...@cloudera.com>:
> On Fri, Oct 24, 2014 at 8:59 PM, Koert Kuipers <ko...@tresata.com> wrote:
> > "mvn clean package -DskipTests" takes about 30 mins for me. thats painful
> > since its needed for the tests. does anyone know any tricks to speed it
> up?
> > (besides getting a better laptop). does zinc help?
>
> Zinc helps by about 50-100%. Worthwhile for sure. brew install zinc
> and zinc -start
>
> > mvn test runs through the projects until one fails. then it skips the
> rest!
> > since its very likely that i get a failure in some subproject, this means
> > its nearly impossible to do a general test run and get a good sense of
> the
> > status of the project. for example:
>
> You can mvn test -pl [module] to test just one module.
> It will also indicate to you that after a failure you can mvn test -rf
> :[module] to continue where it left off -- you can use this to resume
> at the next module.
>
> Or try "-Dscalatest.testFailureIgnore=true" if the mvn flags
> themselves don't work, for continuing after a test failure.
>
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Reactor Summary:
> > [INFO]
> > [INFO] Spark Project Parent POM .......................... SUCCESS
> [2.199s]
> > [INFO] Spark Project Core ................................ SUCCESS
> > [39:43.028s]
> > [INFO] Spark Project Bagel ............................... SUCCESS
> [42.569s]
> > [INFO] Spark Project GraphX .............................. SUCCESS
> > [3:22.104s]
> > [INFO] Spark Project Streaming ........................... SUCCESS
> > [7:12.592s]
> > [INFO] Spark Project ML Library .......................... SUCCESS
> > [10:32.682s]
> > [INFO] Spark Project Tools ............................... SUCCESS
> [17.070s]
> > [INFO] Spark Project Catalyst ............................ SUCCESS
> > [3:03.470s]
> > [INFO] Spark Project SQL ................................. SUCCESS
> > [5:23.993s]
> > [INFO] Spark Project Hive ................................ FAILURE
> > [2:08.387s]
> > [INFO] Spark Project REPL ................................ SKIPPED
> > [INFO] Spark Project Assembly ............................ SKIPPED
> > [INFO] Spark Project External Twitter .................... SKIPPED
> > [INFO] Spark Project External Kafka ...................... SKIPPED
> > [INFO] Spark Project External Flume Sink ................. SKIPPED
> > [INFO] Spark Project External Flume ...................... SKIPPED
> > [INFO] Spark Project External ZeroMQ ..................... SKIPPED
> > [INFO] Spark Project External MQTT ....................... SKIPPED
> > [INFO] Spark Project Examples ............................ SKIPPED
> >
> > in this case i dont care about Hive, but i would have liked to see REPL
> > run, and Kafka.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: scalastyle annoys me a little bit
Posted by Sean Owen <so...@cloudera.com>.
On Fri, Oct 24, 2014 at 8:59 PM, Koert Kuipers <ko...@tresata.com> wrote:
> "mvn clean package -DskipTests" takes about 30 mins for me. thats painful
> since its needed for the tests. does anyone know any tricks to speed it up?
> (besides getting a better laptop). does zinc help?
Zinc helps by about 50-100%. Worthwhile for sure. brew install zinc
and zinc -start
> mvn test runs through the projects until one fails. then it skips the rest!
> since its very likely that i get a failure in some subproject, this means
> its nearly impossible to do a general test run and get a good sense of the
> status of the project. for example:
You can mvn test -pl [module] to test just one module.
It will also indicate to you that after a failure you can mvn test -rf
:[module] to continue where it left off -- you can use this to resume
at the next module.
Or try "-Dscalatest.testFailureIgnore=true" if the mvn flags
themselves don't work, for continuing after a test failure.
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project Parent POM .......................... SUCCESS [2.199s]
> [INFO] Spark Project Core ................................ SUCCESS
> [39:43.028s]
> [INFO] Spark Project Bagel ............................... SUCCESS [42.569s]
> [INFO] Spark Project GraphX .............................. SUCCESS
> [3:22.104s]
> [INFO] Spark Project Streaming ........................... SUCCESS
> [7:12.592s]
> [INFO] Spark Project ML Library .......................... SUCCESS
> [10:32.682s]
> [INFO] Spark Project Tools ............................... SUCCESS [17.070s]
> [INFO] Spark Project Catalyst ............................ SUCCESS
> [3:03.470s]
> [INFO] Spark Project SQL ................................. SUCCESS
> [5:23.993s]
> [INFO] Spark Project Hive ................................ FAILURE
> [2:08.387s]
> [INFO] Spark Project REPL ................................ SKIPPED
> [INFO] Spark Project Assembly ............................ SKIPPED
> [INFO] Spark Project External Twitter .................... SKIPPED
> [INFO] Spark Project External Kafka ...................... SKIPPED
> [INFO] Spark Project External Flume Sink ................. SKIPPED
> [INFO] Spark Project External Flume ...................... SKIPPED
> [INFO] Spark Project External ZeroMQ ..................... SKIPPED
> [INFO] Spark Project External MQTT ....................... SKIPPED
> [INFO] Spark Project Examples ............................ SKIPPED
>
> in this case i dont care about Hive, but i would have liked to see REPL
> run, and Kafka.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: scalastyle annoys me a little bit
Posted by Marcelo Vanzin <va...@cloudera.com>.
On Fri, Oct 24, 2014 at 12:59 PM, Koert Kuipers <ko...@tresata.com> wrote:
> "mvn clean package -DskipTests" takes about 30 mins for me. thats painful
> since its needed for the tests. does anyone know any tricks to speed it up?
> (besides getting a better laptop). does zinc help?
I noticed this too, and I also noticed some messages about running out
of space for the code cache on the output. In the pom.xml I added this
to the scala compiler options:
<jvmArg>-XX:ReservedCodeCacheSize=512m</jvmArg>
Although I haven't actually measured if it improves things yet.
Haven't tried zinc either.
> mvn test runs through the projects until one fails. then it skips the rest!
> since its very likely that i get a failure in some subproject, this means
> its nearly impossible to do a general test run and get a good sense of the
> status of the project. for example:
You can try "mvn -fn" or "mvn -fae" (check "mvn --help" for what they mean).
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: scalastyle annoys me a little bit
Posted by Koert Kuipers <ko...@tresata.com>.
thanks ted.
apologies for complaining about maven here again, but this is the first
time i seriously use it for development, and i am completely unfamiliar
with it.
a few more issues:
"mvn clean package -DskipTests" takes about 30 mins for me. thats painful
since its needed for the tests. does anyone know any tricks to speed it up?
(besides getting a better laptop). does zinc help?
does scalastyle overwrite files? after i do "mvn package" emacs is
completely confused and for every file it says "it has been edited" and i
need to re-open it. not helpful for a development cycle. i think i am
simply going to edit the pom and remove scalacheck for development.
mvn test runs through the projects until one fails. then it skips the rest!
since its very likely that i get a failure in some subproject, this means
its nearly impossible to do a general test run and get a good sense of the
status of the project. for example:
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM .......................... SUCCESS [2.199s]
[INFO] Spark Project Core ................................ SUCCESS
[39:43.028s]
[INFO] Spark Project Bagel ............................... SUCCESS [42.569s]
[INFO] Spark Project GraphX .............................. SUCCESS
[3:22.104s]
[INFO] Spark Project Streaming ........................... SUCCESS
[7:12.592s]
[INFO] Spark Project ML Library .......................... SUCCESS
[10:32.682s]
[INFO] Spark Project Tools ............................... SUCCESS [17.070s]
[INFO] Spark Project Catalyst ............................ SUCCESS
[3:03.470s]
[INFO] Spark Project SQL ................................. SUCCESS
[5:23.993s]
[INFO] Spark Project Hive ................................ FAILURE
[2:08.387s]
[INFO] Spark Project REPL ................................ SKIPPED
[INFO] Spark Project Assembly ............................ SKIPPED
[INFO] Spark Project External Twitter .................... SKIPPED
[INFO] Spark Project External Kafka ...................... SKIPPED
[INFO] Spark Project External Flume Sink ................. SKIPPED
[INFO] Spark Project External Flume ...................... SKIPPED
[INFO] Spark Project External ZeroMQ ..................... SKIPPED
[INFO] Spark Project External MQTT ....................... SKIPPED
[INFO] Spark Project Examples ............................ SKIPPED
in this case i dont care about Hive, but i would have liked to see REPL
run, and Kafka.
On Thu, Oct 23, 2014 at 4:44 PM, Ted Yu <yu...@gmail.com> wrote:
> Created SPARK-4066 and attached patch there.
>
> On Thu, Oct 23, 2014 at 1:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
>
>> great thanks i will do that
>>
>> On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Koert:
>>> If you have time, you can try this diff - with which you would be able
>>> to specify the following on the command line:
>>> -Dscalastyle.failonviolation=false
>>>
>>> diff --git a/pom.xml b/pom.xml
>>> index 687cc63..108585e 100644
>>> --- a/pom.xml
>>> +++ b/pom.xml
>>> @@ -123,6 +123,7 @@
>>> <log4j.version>1.2.17</log4j.version>
>>> <hadoop.version>1.0.4</hadoop.version>
>>> <protobuf.version>2.4.1</protobuf.version>
>>> + <scalastyle.failonviolation>true</scalastyle.failonviolation>
>>> <yarn.version>${hadoop.version}</yarn.version>
>>> <hbase.version>0.94.6</hbase.version>
>>> <flume.version>1.4.0</flume.version>
>>> @@ -1071,7 +1072,7 @@
>>> <version>0.4.0</version>
>>> <configuration>
>>> <verbose>false</verbose>
>>> - <failOnViolation>true</failOnViolation>
>>> +
>>> <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
>>> <includeTestSourceDirectory>false</includeTestSourceDirectory>
>>> <failOnWarning>false</failOnWarning>
>>> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
>>>
>>>
>>>
>>> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com>
>>> wrote:
>>>
>>>> Hey Ted,
>>>> i tried:
>>>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>>>>
>>>> no luck, still get
>>>> [ERROR] Failed to execute goal
>>>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
>>>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
>>>> violation(s). -> [Help 1]
>>>>
>>>>
>>>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>
>>>>> Koert:
>>>>> Have you tried adding the following on your commandline ?
>>>>>
>>>>> -Dscalastyle.failOnViolation=false
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pw...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hey Koert,
>>>>>>
>>>>>> I think disabling the style checks in maven package could be a good
>>>>>> idea for the reason you point out. I was sort of mixed on that when it
>>>>>> was proposed for this exact reason. It's just annoying to developers.
>>>>>>
>>>>>> In terms of changing the global limit, this is more religion than
>>>>>> anything else, but there are other cases where the current limit is
>>>>>> useful (e.g. if you have many windows open in a large screen).
>>>>>>
>>>>>> - Patrick
>>>>>>
>>>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>>>>> wrote:
>>>>>> > 100 max width seems very restrictive to me.
>>>>>> >
>>>>>> > even the most restrictive environment i have for development (ssh
>>>>>> with
>>>>>> > emacs) i get a lot more characters to work with than that.
>>>>>> >
>>>>>> > personally i find the code harder to read, not easier. like i kept
>>>>>> > wondering why there are weird newlines in the
>>>>>> > middle of constructors and such, only to realise later it was
>>>>>> because of
>>>>>> > the 100 character limit.
>>>>>> >
>>>>>> > also, i find "mvn package" erroring out because of style errors
>>>>>> somewhat
>>>>>> > excessive. i understand that a pull request needs to conform to
>>>>>> "the style"
>>>>>> > before being accepted, but this means i cant even run tests on code
>>>>>> that
>>>>>> > does not conform to the style guide, which is a bit silly.
>>>>>> >
>>>>>> > i keep going out for coffee while package and tests run, only to
>>>>>> come back
>>>>>> > for an annoying error that my line is 101 characters and therefore
>>>>>> nothing
>>>>>> > ran.
>>>>>> >
>>>>>> > is there some maven switch to disable the style checks?
>>>>>> >
>>>>>> > best! koert
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Ted Yu <yu...@gmail.com>.
Created SPARK-4066 and attached patch there.
On Thu, Oct 23, 2014 at 1:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
> great thanks i will do that
>
> On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Koert:
>> If you have time, you can try this diff - with which you would be able to
>> specify the following on the command line:
>> -Dscalastyle.failonviolation=false
>>
>> diff --git a/pom.xml b/pom.xml
>> index 687cc63..108585e 100644
>> --- a/pom.xml
>> +++ b/pom.xml
>> @@ -123,6 +123,7 @@
>> <log4j.version>1.2.17</log4j.version>
>> <hadoop.version>1.0.4</hadoop.version>
>> <protobuf.version>2.4.1</protobuf.version>
>> + <scalastyle.failonviolation>true</scalastyle.failonviolation>
>> <yarn.version>${hadoop.version}</yarn.version>
>> <hbase.version>0.94.6</hbase.version>
>> <flume.version>1.4.0</flume.version>
>> @@ -1071,7 +1072,7 @@
>> <version>0.4.0</version>
>> <configuration>
>> <verbose>false</verbose>
>> - <failOnViolation>true</failOnViolation>
>> +
>> <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
>> <includeTestSourceDirectory>false</includeTestSourceDirectory>
>> <failOnWarning>false</failOnWarning>
>> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
>>
>>
>>
>> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com>
>> wrote:
>>
>>> Hey Ted,
>>> i tried:
>>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>>>
>>> no luck, still get
>>> [ERROR] Failed to execute goal
>>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
>>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
>>> violation(s). -> [Help 1]
>>>
>>>
>>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yu...@gmail.com> wrote:
>>>
>>>> Koert:
>>>> Have you tried adding the following on your commandline ?
>>>>
>>>> -Dscalastyle.failOnViolation=false
>>>>
>>>> Cheers
>>>>
>>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pw...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hey Koert,
>>>>>
>>>>> I think disabling the style checks in maven package could be a good
>>>>> idea for the reason you point out. I was sort of mixed on that when it
>>>>> was proposed for this exact reason. It's just annoying to developers.
>>>>>
>>>>> In terms of changing the global limit, this is more religion than
>>>>> anything else, but there are other cases where the current limit is
>>>>> useful (e.g. if you have many windows open in a large screen).
>>>>>
>>>>> - Patrick
>>>>>
>>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>>>> wrote:
>>>>> > 100 max width seems very restrictive to me.
>>>>> >
>>>>> > even the most restrictive environment i have for development (ssh
>>>>> with
>>>>> > emacs) i get a lot more characters to work with than that.
>>>>> >
>>>>> > personally i find the code harder to read, not easier. like i kept
>>>>> > wondering why there are weird newlines in the
>>>>> > middle of constructors and such, only to realise later it was
>>>>> because of
>>>>> > the 100 character limit.
>>>>> >
>>>>> > also, i find "mvn package" erroring out because of style errors
>>>>> somewhat
>>>>> > excessive. i understand that a pull request needs to conform to "the
>>>>> style"
>>>>> > before being accepted, but this means i cant even run tests on code
>>>>> that
>>>>> > does not conform to the style guide, which is a bit silly.
>>>>> >
>>>>> > i keep going out for coffee while package and tests run, only to
>>>>> come back
>>>>> > for an annoying error that my line is 101 characters and therefore
>>>>> nothing
>>>>> > ran.
>>>>> >
>>>>> > is there some maven switch to disable the style checks?
>>>>> >
>>>>> > best! koert
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Koert Kuipers <ko...@tresata.com>.
great thanks i will do that
On Thu, Oct 23, 2014 at 3:55 PM, Ted Yu <yu...@gmail.com> wrote:
> Koert:
> If you have time, you can try this diff - with which you would be able to
> specify the following on the command line:
> -Dscalastyle.failonviolation=false
>
> diff --git a/pom.xml b/pom.xml
> index 687cc63..108585e 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -123,6 +123,7 @@
> <log4j.version>1.2.17</log4j.version>
> <hadoop.version>1.0.4</hadoop.version>
> <protobuf.version>2.4.1</protobuf.version>
> + <scalastyle.failonviolation>true</scalastyle.failonviolation>
> <yarn.version>${hadoop.version}</yarn.version>
> <hbase.version>0.94.6</hbase.version>
> <flume.version>1.4.0</flume.version>
> @@ -1071,7 +1072,7 @@
> <version>0.4.0</version>
> <configuration>
> <verbose>false</verbose>
> - <failOnViolation>true</failOnViolation>
> + <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
> <includeTestSourceDirectory>false</includeTestSourceDirectory>
> <failOnWarning>false</failOnWarning>
> <sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
>
>
>
> On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
>
>> Hey Ted,
>> i tried:
>> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>>
>> no luck, still get
>> [ERROR] Failed to execute goal
>> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
>> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
>> violation(s). -> [Help 1]
>>
>>
>> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Koert:
>>> Have you tried adding the following on your commandline ?
>>>
>>> -Dscalastyle.failOnViolation=false
>>>
>>> Cheers
>>>
>>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pw...@gmail.com>
>>> wrote:
>>>
>>>> Hey Koert,
>>>>
>>>> I think disabling the style checks in maven package could be a good
>>>> idea for the reason you point out. I was sort of mixed on that when it
>>>> was proposed for this exact reason. It's just annoying to developers.
>>>>
>>>> In terms of changing the global limit, this is more religion than
>>>> anything else, but there are other cases where the current limit is
>>>> useful (e.g. if you have many windows open in a large screen).
>>>>
>>>> - Patrick
>>>>
>>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>>> wrote:
>>>> > 100 max width seems very restrictive to me.
>>>> >
>>>> > even the most restrictive environment i have for development (ssh with
>>>> > emacs) i get a lot more characters to work with than that.
>>>> >
>>>> > personally i find the code harder to read, not easier. like i kept
>>>> > wondering why there are weird newlines in the
>>>> > middle of constructors and such, only to realise later it was because
>>>> of
>>>> > the 100 character limit.
>>>> >
>>>> > also, i find "mvn package" erroring out because of style errors
>>>> somewhat
>>>> > excessive. i understand that a pull request needs to conform to "the
>>>> style"
>>>> > before being accepted, but this means i cant even run tests on code
>>>> that
>>>> > does not conform to the style guide, which is a bit silly.
>>>> >
>>>> > i keep going out for coffee while package and tests run, only to come
>>>> back
>>>> > for an annoying error that my line is 101 characters and therefore
>>>> nothing
>>>> > ran.
>>>> >
>>>> > is there some maven switch to disable the style checks?
>>>> >
>>>> > best! koert
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>>
>>>>
>>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Ted Yu <yu...@gmail.com>.
Koert:
If you have time, you can try this diff - with which you would be able to
specify the following on the command line:
-Dscalastyle.failonviolation=false
diff --git a/pom.xml b/pom.xml
index 687cc63..108585e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -123,6 +123,7 @@
<log4j.version>1.2.17</log4j.version>
<hadoop.version>1.0.4</hadoop.version>
<protobuf.version>2.4.1</protobuf.version>
+ <scalastyle.failonviolation>true</scalastyle.failonviolation>
<yarn.version>${hadoop.version}</yarn.version>
<hbase.version>0.94.6</hbase.version>
<flume.version>1.4.0</flume.version>
@@ -1071,7 +1072,7 @@
<version>0.4.0</version>
<configuration>
<verbose>false</verbose>
- <failOnViolation>true</failOnViolation>
+ <failOnViolation>${scalastyle.failonviolation}</failOnViolation>
<includeTestSourceDirectory>false</includeTestSourceDirectory>
<failOnWarning>false</failOnWarning>
<sourceDirectory>${basedir}/src/main/scala</sourceDirectory>
On Thu, Oct 23, 2014 at 12:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
> Hey Ted,
> i tried:
> mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
>
> no luck, still get
> [ERROR] Failed to execute goal
> org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
> spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
> violation(s). -> [Help 1]
>
>
> On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Koert:
>> Have you tried adding the following on your commandline ?
>>
>> -Dscalastyle.failOnViolation=false
>>
>> Cheers
>>
>> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pw...@gmail.com>
>> wrote:
>>
>>> Hey Koert,
>>>
>>> I think disabling the style checks in maven package could be a good
>>> idea for the reason you point out. I was sort of mixed on that when it
>>> was proposed for this exact reason. It's just annoying to developers.
>>>
>>> In terms of changing the global limit, this is more religion than
>>> anything else, but there are other cases where the current limit is
>>> useful (e.g. if you have many windows open in a large screen).
>>>
>>> - Patrick
>>>
>>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>>> wrote:
>>> > 100 max width seems very restrictive to me.
>>> >
>>> > even the most restrictive environment i have for development (ssh with
>>> > emacs) i get a lot more characters to work with than that.
>>> >
>>> > personally i find the code harder to read, not easier. like i kept
>>> > wondering why there are weird newlines in the
>>> > middle of constructors and such, only to realise later it was because
>>> of
>>> > the 100 character limit.
>>> >
>>> > also, i find "mvn package" erroring out because of style errors
>>> somewhat
>>> > excessive. i understand that a pull request needs to conform to "the
>>> style"
>>> > before being accepted, but this means i cant even run tests on code
>>> that
>>> > does not conform to the style guide, which is a bit silly.
>>> >
>>> > i keep going out for coffee while package and tests run, only to come
>>> back
>>> > for an annoying error that my line is 101 characters and therefore
>>> nothing
>>> > ran.
>>> >
>>> > is there some maven switch to disable the style checks?
>>> >
>>> > best! koert
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>
>>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Koert Kuipers <ko...@tresata.com>.
Hey Ted,
i tried:
mvn clean package -DskipTests -Dscalastyle.failOnViolation=false
no luck, still get
[ERROR] Failed to execute goal
org.scalastyle:scalastyle-maven-plugin:0.4.0:check (default) on project
spark-core_2.10: Failed during scalastyle execution: You have 3 Scalastyle
violation(s). -> [Help 1]
On Thu, Oct 23, 2014 at 2:14 PM, Ted Yu <yu...@gmail.com> wrote:
> Koert:
> Have you tried adding the following on your commandline ?
>
> -Dscalastyle.failOnViolation=false
>
> Cheers
>
> On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pw...@gmail.com>
> wrote:
>
>> Hey Koert,
>>
>> I think disabling the style checks in maven package could be a good
>> idea for the reason you point out. I was sort of mixed on that when it
>> was proposed for this exact reason. It's just annoying to developers.
>>
>> In terms of changing the global limit, this is more religion than
>> anything else, but there are other cases where the current limit is
>> useful (e.g. if you have many windows open in a large screen).
>>
>> - Patrick
>>
>> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com>
>> wrote:
>> > 100 max width seems very restrictive to me.
>> >
>> > even the most restrictive environment i have for development (ssh with
>> > emacs) i get a lot more characters to work with than that.
>> >
>> > personally i find the code harder to read, not easier. like i kept
>> > wondering why there are weird newlines in the
>> > middle of constructors and such, only to realise later it was because of
>> > the 100 character limit.
>> >
>> > also, i find "mvn package" erroring out because of style errors somewhat
>> > excessive. i understand that a pull request needs to conform to "the
>> style"
>> > before being accepted, but this means i cant even run tests on code that
>> > does not conform to the style guide, which is a bit silly.
>> >
>> > i keep going out for coffee while package and tests run, only to come
>> back
>> > for an annoying error that my line is 101 characters and therefore
>> nothing
>> > ran.
>> >
>> > is there some maven switch to disable the style checks?
>> >
>> > best! koert
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
Re: scalastyle annoys me a little bit
Posted by Ted Yu <yu...@gmail.com>.
Koert:
Have you tried adding the following on your commandline ?
-Dscalastyle.failOnViolation=false
Cheers
On Thu, Oct 23, 2014 at 11:07 AM, Patrick Wendell <pw...@gmail.com>
wrote:
> Hey Koert,
>
> I think disabling the style checks in maven package could be a good
> idea for the reason you point out. I was sort of mixed on that when it
> was proposed for this exact reason. It's just annoying to developers.
>
> In terms of changing the global limit, this is more religion than
> anything else, but there are other cases where the current limit is
> useful (e.g. if you have many windows open in a large screen).
>
> - Patrick
>
> On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > 100 max width seems very restrictive to me.
> >
> > even the most restrictive environment i have for development (ssh with
> > emacs) i get a lot more characters to work with than that.
> >
> > personally i find the code harder to read, not easier. like i kept
> > wondering why there are weird newlines in the
> > middle of constructors and such, only to realise later it was because of
> > the 100 character limit.
> >
> > also, i find "mvn package" erroring out because of style errors somewhat
> > excessive. i understand that a pull request needs to conform to "the
> style"
> > before being accepted, but this means i cant even run tests on code that
> > does not conform to the style guide, which is a bit silly.
> >
> > i keep going out for coffee while package and tests run, only to come
> back
> > for an annoying error that my line is 101 characters and therefore
> nothing
> > ran.
> >
> > is there some maven switch to disable the style checks?
> >
> > best! koert
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: scalastyle annoys me a little bit
Posted by Patrick Wendell <pw...@gmail.com>.
Hey Koert,
I think disabling the style checks in maven package could be a good
idea for the reason you point out. I was sort of mixed on that when it
was proposed for this exact reason. It's just annoying to developers.
In terms of changing the global limit, this is more religion than
anything else, but there are other cases where the current limit is
useful (e.g. if you have many windows open in a large screen).
- Patrick
On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com> wrote:
> 100 max width seems very restrictive to me.
>
> even the most restrictive environment i have for development (ssh with
> emacs) i get a lot more characters to work with than that.
>
> personally i find the code harder to read, not easier. like i kept
> wondering why there are weird newlines in the
> middle of constructors and such, only to realise later it was because of
> the 100 character limit.
>
> also, i find "mvn package" erroring out because of style errors somewhat
> excessive. i understand that a pull request needs to conform to "the style"
> before being accepted, but this means i cant even run tests on code that
> does not conform to the style guide, which is a bit silly.
>
> i keep going out for coffee while package and tests run, only to come back
> for an annoying error that my line is 101 characters and therefore nothing
> ran.
>
> is there some maven switch to disable the style checks?
>
> best! koert
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: scalastyle annoys me a little bit
Posted by Marcelo Vanzin <va...@cloudera.com>.
I know this is all very subjective, but I find long lines difficult to read.
I also like how 100 characters fit in my editor setup fine (split wide
screen), while a longer line length would mean I can't have two
buffers side-by-side without horizontal scrollbars.
I think it's fine to add a switch to skip the style tests, but then,
you'll still have to fix the issues at some point...
On Thu, Oct 23, 2014 at 11:03 AM, Koert Kuipers <ko...@tresata.com> wrote:
> 100 max width seems very restrictive to me.
>
> even the most restrictive environment i have for development (ssh with
> emacs) i get a lot more characters to work with than that.
>
> personally i find the code harder to read, not easier. like i kept
> wondering why there are weird newlines in the
> middle of constructors and such, only to realise later it was because of
> the 100 character limit.
>
> also, i find "mvn package" erroring out because of style errors somewhat
> excessive. i understand that a pull request needs to conform to "the style"
> before being accepted, but this means i cant even run tests on code that
> does not conform to the style guide, which is a bit silly.
>
> i keep going out for coffee while package and tests run, only to come back
> for an annoying error that my line is 101 characters and therefore nothing
> ran.
>
> is there some maven switch to disable the style checks?
>
> best! koert
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org