You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Nicholas Chammas <ni...@gmail.com> on 2014/10/01 23:01:06 UTC
Extending Scala style checks
As discussed here <https://github.com/apache/spark/pull/2619>, it would be
good to extend our Scala style checks to programmatically enforce as many
of our style rules as possible.
Does anyone know if it's relatively straightforward to enforce additional
rules like the "no trailing spaces" rule mentioned in the linked PR?
Nick
Re: Extending Scala style checks
Posted by Reynold Xin <rx...@databricks.com>.
There is scalariform but it can be disruptive. Last time I ran it on Spark
it didn't compile due to some xml interpolation problem.
On Wednesday, October 1, 2014, Nicholas Chammas <ni...@gmail.com>
wrote:
> Does anyone know if Scala has something equivalent to autopep8
> <https://pypi.python.org/pypi/autopep8>? It would help patch up the
> existing code base a lot quicker as we add in new style rules.
>
>
> On Wed, Oct 1, 2014 at 9:24 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com <javascript:;>
> > wrote:
>
> > Yeah, I remember that hell when I added PEP 8 to the build checks and
> > fixed all the outstanding Python style issues. I had to keep rebasing and
> > resolving merge conflicts until the PR was merged.
> >
> > It's a rough process, but thankfully it's also a one-time process. I
> might
> > be able to help with that in the next week or two if no-one else wants to
> > pick it up.
> >
> > Nick
> >
> > On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <michael@databricks.com
> <javascript:;>>
> > wrote:
> >
> >> The hard part here is updating the existing code base... which is going
> >> to create merge conflicts with like all of the open PRs...
> >>
> >> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
> >> nicholas.chammas@gmail.com <javascript:;>> wrote:
> >>
> >>> Ah, since there appears to be a built-in rule for end-of-line
> whitespace,
> >>> Michael and Cheng, y'all should be able to add this in pretty easily.
> >>>
> >>> Nick
> >>>
> >>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pwendell@gmail.com
> <javascript:;>>
> >>> wrote:
> >>>
> >>> > Hey Nick,
> >>> >
> >>> > We can always take built-in rules. Back when we added this Prashant
> >>> > Sharma actually did some great work that lets us write our own style
> >>> > rules in cases where rules don't exist.
> >>> >
> >>> > You can see some existing rules here:
> >>> >
> >>> >
> >>>
> https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
> >>> >
> >>> > Prashant has over time contributed a lot of our custom rules upstream
> >>> > to stalastyle, so now there are only a couple there.
> >>> >
> >>> > - Patrick
> >>> >
> >>> > On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yuzhihong@gmail.com
> <javascript:;>> wrote:
> >>> > > Please take a look at WhitespaceEndOfLineChecker under:
> >>> > > http://www.scalastyle.org/rules-0.1.0.html
> >>> > >
> >>> > > Cheers
> >>> > >
> >>> > > On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
> >>> > nicholas.chammas@gmail.com <javascript:;>
> >>> > >> wrote:
> >>> > >
> >>> > >> As discussed here <https://github.com/apache/spark/pull/2619>, it
> >>> > would be
> >>> > >> good to extend our Scala style checks to programmatically enforce
> as
> >>> > many
> >>> > >> of our style rules as possible.
> >>> > >>
> >>> > >> Does anyone know if it's relatively straightforward to enforce
> >>> > additional
> >>> > >> rules like the "no trailing spaces" rule mentioned in the linked
> PR?
> >>> > >>
> >>> > >> Nick
> >>> > >>
> >>> >
> >>>
> >>
> >>
> >
>
Re: Extending Scala style checks
Posted by Nicholas Chammas <ni...@gmail.com>.
Does anyone know if Scala has something equivalent to autopep8
<https://pypi.python.org/pypi/autopep8>? It would help patch up the
existing code base a lot quicker as we add in new style rules.
On Wed, Oct 1, 2014 at 9:24 PM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> Yeah, I remember that hell when I added PEP 8 to the build checks and
> fixed all the outstanding Python style issues. I had to keep rebasing and
> resolving merge conflicts until the PR was merged.
>
> It's a rough process, but thankfully it's also a one-time process. I might
> be able to help with that in the next week or two if no-one else wants to
> pick it up.
>
> Nick
>
> On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
>> The hard part here is updating the existing code base... which is going
>> to create merge conflicts with like all of the open PRs...
>>
>> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
>> nicholas.chammas@gmail.com> wrote:
>>
>>> Ah, since there appears to be a built-in rule for end-of-line whitespace,
>>> Michael and Cheng, y'all should be able to add this in pretty easily.
>>>
>>> Nick
>>>
>>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
>>> wrote:
>>>
>>> > Hey Nick,
>>> >
>>> > We can always take built-in rules. Back when we added this Prashant
>>> > Sharma actually did some great work that lets us write our own style
>>> > rules in cases where rules don't exist.
>>> >
>>> > You can see some existing rules here:
>>> >
>>> >
>>> https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
>>> >
>>> > Prashant has over time contributed a lot of our custom rules upstream
>>> > to stalastyle, so now there are only a couple there.
>>> >
>>> > - Patrick
>>> >
>>> > On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
>>> > > Please take a look at WhitespaceEndOfLineChecker under:
>>> > > http://www.scalastyle.org/rules-0.1.0.html
>>> > >
>>> > > Cheers
>>> > >
>>> > > On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
>>> > nicholas.chammas@gmail.com
>>> > >> wrote:
>>> > >
>>> > >> As discussed here <https://github.com/apache/spark/pull/2619>, it
>>> > would be
>>> > >> good to extend our Scala style checks to programmatically enforce as
>>> > many
>>> > >> of our style rules as possible.
>>> > >>
>>> > >> Does anyone know if it's relatively straightforward to enforce
>>> > additional
>>> > >> rules like the "no trailing spaces" rule mentioned in the linked PR?
>>> > >>
>>> > >> Nick
>>> > >>
>>> >
>>>
>>
>>
>
Re: Extending Scala style checks
Posted by Nicholas Chammas <ni...@gmail.com>.
FYI: scalastyle just merged in a patch to add support for external rules
<https://github.com/scalastyle/scalastyle/issues/25#ref-commit-9a576c3>.
I forget why I was following the linked issue, but I assume it's related to
this discussion.
Nick
On Thu Oct 09 2014 at 2:56:30 AM Reynold Xin <rx...@databricks.com> wrote:
> Thanks. I added one.
>
>
> On Wed, Oct 8, 2014 at 8:49 AM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> I've created SPARK-3849: Automate remaining Scala style rules
>>
> <https://issues.apache.org/jira/browse/SPARK-3849>.
>
>
>>
>> Please create sub-tasks on this issue for rules that we have not automated
>> and let's work through them as possible.
>>
>> I went ahead and created the first sub-task, SPARK-3850: Scala style:
>>
> Disallow trailing spaces <https://issues.apache.org/jira/browse/SPARK-3850
>> >.
>
>
>>
>> Nick
>>
>> On Tue, Oct 7, 2014 at 4:45 PM, Nicholas Chammas <
>> nicholas.chammas@gmail.com
>> > wrote:
>>
>> > For starters, do we have a list of all the Scala style rules that are
>> > currently not enforced automatically but are likely well-suited for
>> > automation?
>> >
>> > Let's put such a list together in a JIRA issue and work through
>> > implementing them.
>> >
>> > Nick
>> >
>> > On Thu, Oct 2, 2014 at 12:06 AM, Cheng Lian <li...@gmail.com>
>> wrote:
>> >
>> >> Since we can easily catch the list of all changed files in a PR, I
>> think
>> >> we can start with adding the no trailing space check for newly changed
>> >> files only?
>> >>
>> >>
>> >> On 10/2/14 9:24 AM, Nicholas Chammas wrote:
>> >>
>> >>> Yeah, I remember that hell when I added PEP 8 to the build checks and
>> >>> fixed
>> >>> all the outstanding Python style issues. I had to keep rebasing and
>> >>> resolving merge conflicts until the PR was merged.
>> >>>
>> >>> It's a rough process, but thankfully it's also a one-time process. I
>> >>> might
>> >>> be able to help with that in the next week or two if no-one else
>> wants to
>> >>> pick it up.
>> >>>
>> >>> Nick
>> >>>
>> >>> On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <
>> michael@databricks.com
>> >>> >
>> >>> wrote:
>> >>>
>> >>> The hard part here is updating the existing code base... which is
>> going
>> >>>> to
>> >>>> create merge conflicts with like all of the open PRs...
>> >>>>
>> >>>> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
>> >>>> nicholas.chammas@gmail.com> wrote:
>> >>>>
>> >>>> Ah, since there appears to be a built-in rule for end-of-line
>> >>>>> whitespace,
>> >>>>> Michael and Cheng, y'all should be able to add this in pretty
>> easily.
>> >>>>>
>> >>>>> Nick
>> >>>>>
>> >>>>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pwendell@gmail.com
>> >
>> >>>>> wrote:
>> >>>>>
>> >>>>> Hey Nick,
>> >>>>>>
>> >>>>>> We can always take built-in rules. Back when we added this Prashant
>> >>>>>> Sharma actually did some great work that lets us write our own
>> style
>> >>>>>> rules in cases where rules don't exist.
>> >>>>>>
>> >>>>>> You can see some existing rules here:
>> >>>>>>
>> >>>>>>
>> >>>>>> https://github.com/apache/spark/tree/master/project/
>> >>>>> spark-style/src/main/scala/org/apache/spark/scalastyle
>> >>>>>
>> >>>>>> Prashant has over time contributed a lot of our custom rules
>> upstream
>> >>>>>> to stalastyle, so now there are only a couple there.
>> >>>>>>
>> >>>>>> - Patrick
>> >>>>>>
>> >>>>>> On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com>
>> wrote:
>> >>>>>>
>> >>>>>>> Please take a look at WhitespaceEndOfLineChecker under:
>> >>>>>>> http://www.scalastyle.org/rules-0.1.0.html
>> >>>>>>>
>> >>>>>>> Cheers
>> >>>>>>>
>> >>>>>>> On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
>> >>>>>>>
>> >>>>>> nicholas.chammas@gmail.com
>> >>>>>>
>> >>>>>>> wrote:
>> >>>>>>>> As discussed here <https://github.com/apache/spark/pull/2619>,
>> it
>> >>>>>>>>
>> >>>>>>> would be
>> >>>>>>
>> >>>>>>> good to extend our Scala style checks to programmatically enforce
>> as
>> >>>>>>>>
>> >>>>>>> many
>> >>>>>>
>> >>>>>>> of our style rules as possible.
>> >>>>>>>>
>> >>>>>>>> Does anyone know if it's relatively straightforward to enforce
>> >>>>>>>>
>> >>>>>>> additional
>> >>>>>>
>> >>>>>>> rules like the "no trailing spaces" rule mentioned in the linked
>> PR?
>> >>>>>>>>
>> >>>>>>>> Nick
>> >>>>>>>>
>> >>>>>>>>
>> >>>>
>> >>
>> >
>>
>
Re: Extending Scala style checks
Posted by Reynold Xin <rx...@databricks.com>.
Thanks. I added one.
On Wed, Oct 8, 2014 at 8:49 AM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> I've created SPARK-3849: Automate remaining Scala style rules
> <https://issues.apache.org/jira/browse/SPARK-3849>.
>
> Please create sub-tasks on this issue for rules that we have not automated
> and let's work through them as possible.
>
> I went ahead and created the first sub-task, SPARK-3850: Scala style:
> Disallow trailing spaces <https://issues.apache.org/jira/browse/SPARK-3850
> >.
>
> Nick
>
> On Tue, Oct 7, 2014 at 4:45 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com
> > wrote:
>
> > For starters, do we have a list of all the Scala style rules that are
> > currently not enforced automatically but are likely well-suited for
> > automation?
> >
> > Let's put such a list together in a JIRA issue and work through
> > implementing them.
> >
> > Nick
> >
> > On Thu, Oct 2, 2014 at 12:06 AM, Cheng Lian <li...@gmail.com>
> wrote:
> >
> >> Since we can easily catch the list of all changed files in a PR, I think
> >> we can start with adding the no trailing space check for newly changed
> >> files only?
> >>
> >>
> >> On 10/2/14 9:24 AM, Nicholas Chammas wrote:
> >>
> >>> Yeah, I remember that hell when I added PEP 8 to the build checks and
> >>> fixed
> >>> all the outstanding Python style issues. I had to keep rebasing and
> >>> resolving merge conflicts until the PR was merged.
> >>>
> >>> It's a rough process, but thankfully it's also a one-time process. I
> >>> might
> >>> be able to help with that in the next week or two if no-one else wants
> to
> >>> pick it up.
> >>>
> >>> Nick
> >>>
> >>> On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <
> michael@databricks.com
> >>> >
> >>> wrote:
> >>>
> >>> The hard part here is updating the existing code base... which is
> going
> >>>> to
> >>>> create merge conflicts with like all of the open PRs...
> >>>>
> >>>> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
> >>>> nicholas.chammas@gmail.com> wrote:
> >>>>
> >>>> Ah, since there appears to be a built-in rule for end-of-line
> >>>>> whitespace,
> >>>>> Michael and Cheng, y'all should be able to add this in pretty easily.
> >>>>>
> >>>>> Nick
> >>>>>
> >>>>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>> Hey Nick,
> >>>>>>
> >>>>>> We can always take built-in rules. Back when we added this Prashant
> >>>>>> Sharma actually did some great work that lets us write our own style
> >>>>>> rules in cases where rules don't exist.
> >>>>>>
> >>>>>> You can see some existing rules here:
> >>>>>>
> >>>>>>
> >>>>>> https://github.com/apache/spark/tree/master/project/
> >>>>> spark-style/src/main/scala/org/apache/spark/scalastyle
> >>>>>
> >>>>>> Prashant has over time contributed a lot of our custom rules
> upstream
> >>>>>> to stalastyle, so now there are only a couple there.
> >>>>>>
> >>>>>> - Patrick
> >>>>>>
> >>>>>> On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
> >>>>>>
> >>>>>>> Please take a look at WhitespaceEndOfLineChecker under:
> >>>>>>> http://www.scalastyle.org/rules-0.1.0.html
> >>>>>>>
> >>>>>>> Cheers
> >>>>>>>
> >>>>>>> On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
> >>>>>>>
> >>>>>> nicholas.chammas@gmail.com
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>> As discussed here <https://github.com/apache/spark/pull/2619>, it
> >>>>>>>>
> >>>>>>> would be
> >>>>>>
> >>>>>>> good to extend our Scala style checks to programmatically enforce
> as
> >>>>>>>>
> >>>>>>> many
> >>>>>>
> >>>>>>> of our style rules as possible.
> >>>>>>>>
> >>>>>>>> Does anyone know if it's relatively straightforward to enforce
> >>>>>>>>
> >>>>>>> additional
> >>>>>>
> >>>>>>> rules like the "no trailing spaces" rule mentioned in the linked
> PR?
> >>>>>>>>
> >>>>>>>> Nick
> >>>>>>>>
> >>>>>>>>
> >>>>
> >>
> >
>
Re: Extending Scala style checks
Posted by Nicholas Chammas <ni...@gmail.com>.
I've created SPARK-3849: Automate remaining Scala style rules
<https://issues.apache.org/jira/browse/SPARK-3849>.
Please create sub-tasks on this issue for rules that we have not automated
and let's work through them as possible.
I went ahead and created the first sub-task, SPARK-3850: Scala style:
Disallow trailing spaces <https://issues.apache.org/jira/browse/SPARK-3850>.
Nick
On Tue, Oct 7, 2014 at 4:45 PM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> For starters, do we have a list of all the Scala style rules that are
> currently not enforced automatically but are likely well-suited for
> automation?
>
> Let's put such a list together in a JIRA issue and work through
> implementing them.
>
> Nick
>
> On Thu, Oct 2, 2014 at 12:06 AM, Cheng Lian <li...@gmail.com> wrote:
>
>> Since we can easily catch the list of all changed files in a PR, I think
>> we can start with adding the no trailing space check for newly changed
>> files only?
>>
>>
>> On 10/2/14 9:24 AM, Nicholas Chammas wrote:
>>
>>> Yeah, I remember that hell when I added PEP 8 to the build checks and
>>> fixed
>>> all the outstanding Python style issues. I had to keep rebasing and
>>> resolving merge conflicts until the PR was merged.
>>>
>>> It's a rough process, but thankfully it's also a one-time process. I
>>> might
>>> be able to help with that in the next week or two if no-one else wants to
>>> pick it up.
>>>
>>> Nick
>>>
>>> On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <michael@databricks.com
>>> >
>>> wrote:
>>>
>>> The hard part here is updating the existing code base... which is going
>>>> to
>>>> create merge conflicts with like all of the open PRs...
>>>>
>>>> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
>>>> nicholas.chammas@gmail.com> wrote:
>>>>
>>>> Ah, since there appears to be a built-in rule for end-of-line
>>>>> whitespace,
>>>>> Michael and Cheng, y'all should be able to add this in pretty easily.
>>>>>
>>>>> Nick
>>>>>
>>>>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hey Nick,
>>>>>>
>>>>>> We can always take built-in rules. Back when we added this Prashant
>>>>>> Sharma actually did some great work that lets us write our own style
>>>>>> rules in cases where rules don't exist.
>>>>>>
>>>>>> You can see some existing rules here:
>>>>>>
>>>>>>
>>>>>> https://github.com/apache/spark/tree/master/project/
>>>>> spark-style/src/main/scala/org/apache/spark/scalastyle
>>>>>
>>>>>> Prashant has over time contributed a lot of our custom rules upstream
>>>>>> to stalastyle, so now there are only a couple there.
>>>>>>
>>>>>> - Patrick
>>>>>>
>>>>>> On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>>>
>>>>>>> Please take a look at WhitespaceEndOfLineChecker under:
>>>>>>> http://www.scalastyle.org/rules-0.1.0.html
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>> On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
>>>>>>>
>>>>>> nicholas.chammas@gmail.com
>>>>>>
>>>>>>> wrote:
>>>>>>>> As discussed here <https://github.com/apache/spark/pull/2619>, it
>>>>>>>>
>>>>>>> would be
>>>>>>
>>>>>>> good to extend our Scala style checks to programmatically enforce as
>>>>>>>>
>>>>>>> many
>>>>>>
>>>>>>> of our style rules as possible.
>>>>>>>>
>>>>>>>> Does anyone know if it's relatively straightforward to enforce
>>>>>>>>
>>>>>>> additional
>>>>>>
>>>>>>> rules like the "no trailing spaces" rule mentioned in the linked PR?
>>>>>>>>
>>>>>>>> Nick
>>>>>>>>
>>>>>>>>
>>>>
>>
>
Re: Extending Scala style checks
Posted by Nicholas Chammas <ni...@gmail.com>.
For starters, do we have a list of all the Scala style rules that are
currently not enforced automatically but are likely well-suited for
automation?
Let's put such a list together in a JIRA issue and work through
implementing them.
Nick
On Thu, Oct 2, 2014 at 12:06 AM, Cheng Lian <li...@gmail.com> wrote:
> Since we can easily catch the list of all changed files in a PR, I think
> we can start with adding the no trailing space check for newly changed
> files only?
>
>
> On 10/2/14 9:24 AM, Nicholas Chammas wrote:
>
>> Yeah, I remember that hell when I added PEP 8 to the build checks and
>> fixed
>> all the outstanding Python style issues. I had to keep rebasing and
>> resolving merge conflicts until the PR was merged.
>>
>> It's a rough process, but thankfully it's also a one-time process. I might
>> be able to help with that in the next week or two if no-one else wants to
>> pick it up.
>>
>> Nick
>>
>> On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <mi...@databricks.com>
>> wrote:
>>
>> The hard part here is updating the existing code base... which is going
>>> to
>>> create merge conflicts with like all of the open PRs...
>>>
>>> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
>>> nicholas.chammas@gmail.com> wrote:
>>>
>>> Ah, since there appears to be a built-in rule for end-of-line
>>>> whitespace,
>>>> Michael and Cheng, y'all should be able to add this in pretty easily.
>>>>
>>>> Nick
>>>>
>>>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
>>>> wrote:
>>>>
>>>> Hey Nick,
>>>>>
>>>>> We can always take built-in rules. Back when we added this Prashant
>>>>> Sharma actually did some great work that lets us write our own style
>>>>> rules in cases where rules don't exist.
>>>>>
>>>>> You can see some existing rules here:
>>>>>
>>>>>
>>>>> https://github.com/apache/spark/tree/master/project/
>>>> spark-style/src/main/scala/org/apache/spark/scalastyle
>>>>
>>>>> Prashant has over time contributed a lot of our custom rules upstream
>>>>> to stalastyle, so now there are only a couple there.
>>>>>
>>>>> - Patrick
>>>>>
>>>>> On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>>
>>>>>> Please take a look at WhitespaceEndOfLineChecker under:
>>>>>> http://www.scalastyle.org/rules-0.1.0.html
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
>>>>>>
>>>>> nicholas.chammas@gmail.com
>>>>>
>>>>>> wrote:
>>>>>>> As discussed here <https://github.com/apache/spark/pull/2619>, it
>>>>>>>
>>>>>> would be
>>>>>
>>>>>> good to extend our Scala style checks to programmatically enforce as
>>>>>>>
>>>>>> many
>>>>>
>>>>>> of our style rules as possible.
>>>>>>>
>>>>>>> Does anyone know if it's relatively straightforward to enforce
>>>>>>>
>>>>>> additional
>>>>>
>>>>>> rules like the "no trailing spaces" rule mentioned in the linked PR?
>>>>>>>
>>>>>>> Nick
>>>>>>>
>>>>>>>
>>>
>
Re: Extending Scala style checks
Posted by Cheng Lian <li...@gmail.com>.
Since we can easily catch the list of all changed files in a PR, I think
we can start with adding the no trailing space check for newly changed
files only?
On 10/2/14 9:24 AM, Nicholas Chammas wrote:
> Yeah, I remember that hell when I added PEP 8 to the build checks and fixed
> all the outstanding Python style issues. I had to keep rebasing and
> resolving merge conflicts until the PR was merged.
>
> It's a rough process, but thankfully it's also a one-time process. I might
> be able to help with that in the next week or two if no-one else wants to
> pick it up.
>
> Nick
>
> On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
>> The hard part here is updating the existing code base... which is going to
>> create merge conflicts with like all of the open PRs...
>>
>> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
>> nicholas.chammas@gmail.com> wrote:
>>
>>> Ah, since there appears to be a built-in rule for end-of-line whitespace,
>>> Michael and Cheng, y'all should be able to add this in pretty easily.
>>>
>>> Nick
>>>
>>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
>>> wrote:
>>>
>>>> Hey Nick,
>>>>
>>>> We can always take built-in rules. Back when we added this Prashant
>>>> Sharma actually did some great work that lets us write our own style
>>>> rules in cases where rules don't exist.
>>>>
>>>> You can see some existing rules here:
>>>>
>>>>
>>> https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
>>>> Prashant has over time contributed a lot of our custom rules upstream
>>>> to stalastyle, so now there are only a couple there.
>>>>
>>>> - Patrick
>>>>
>>>> On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>> Please take a look at WhitespaceEndOfLineChecker under:
>>>>> http://www.scalastyle.org/rules-0.1.0.html
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
>>>> nicholas.chammas@gmail.com
>>>>>> wrote:
>>>>>> As discussed here <https://github.com/apache/spark/pull/2619>, it
>>>> would be
>>>>>> good to extend our Scala style checks to programmatically enforce as
>>>> many
>>>>>> of our style rules as possible.
>>>>>>
>>>>>> Does anyone know if it's relatively straightforward to enforce
>>>> additional
>>>>>> rules like the "no trailing spaces" rule mentioned in the linked PR?
>>>>>>
>>>>>> Nick
>>>>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Extending Scala style checks
Posted by Nicholas Chammas <ni...@gmail.com>.
Yeah, I remember that hell when I added PEP 8 to the build checks and fixed
all the outstanding Python style issues. I had to keep rebasing and
resolving merge conflicts until the PR was merged.
It's a rough process, but thankfully it's also a one-time process. I might
be able to help with that in the next week or two if no-one else wants to
pick it up.
Nick
On Wed, Oct 1, 2014 at 9:20 PM, Michael Armbrust <mi...@databricks.com>
wrote:
> The hard part here is updating the existing code base... which is going to
> create merge conflicts with like all of the open PRs...
>
> On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> Ah, since there appears to be a built-in rule for end-of-line whitespace,
>> Michael and Cheng, y'all should be able to add this in pretty easily.
>>
>> Nick
>>
>> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
>> wrote:
>>
>> > Hey Nick,
>> >
>> > We can always take built-in rules. Back when we added this Prashant
>> > Sharma actually did some great work that lets us write our own style
>> > rules in cases where rules don't exist.
>> >
>> > You can see some existing rules here:
>> >
>> >
>> https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
>> >
>> > Prashant has over time contributed a lot of our custom rules upstream
>> > to stalastyle, so now there are only a couple there.
>> >
>> > - Patrick
>> >
>> > On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
>> > > Please take a look at WhitespaceEndOfLineChecker under:
>> > > http://www.scalastyle.org/rules-0.1.0.html
>> > >
>> > > Cheers
>> > >
>> > > On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
>> > nicholas.chammas@gmail.com
>> > >> wrote:
>> > >
>> > >> As discussed here <https://github.com/apache/spark/pull/2619>, it
>> > would be
>> > >> good to extend our Scala style checks to programmatically enforce as
>> > many
>> > >> of our style rules as possible.
>> > >>
>> > >> Does anyone know if it's relatively straightforward to enforce
>> > additional
>> > >> rules like the "no trailing spaces" rule mentioned in the linked PR?
>> > >>
>> > >> Nick
>> > >>
>> >
>>
>
>
Re: Extending Scala style checks
Posted by Michael Armbrust <mi...@databricks.com>.
The hard part here is updating the existing code base... which is going to
create merge conflicts with like all of the open PRs...
On Wed, Oct 1, 2014 at 6:13 PM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> Ah, since there appears to be a built-in rule for end-of-line whitespace,
> Michael and Cheng, y'all should be able to add this in pretty easily.
>
> Nick
>
> On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
> wrote:
>
> > Hey Nick,
> >
> > We can always take built-in rules. Back when we added this Prashant
> > Sharma actually did some great work that lets us write our own style
> > rules in cases where rules don't exist.
> >
> > You can see some existing rules here:
> >
> >
> https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
> >
> > Prashant has over time contributed a lot of our custom rules upstream
> > to stalastyle, so now there are only a couple there.
> >
> > - Patrick
> >
> > On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
> > > Please take a look at WhitespaceEndOfLineChecker under:
> > > http://www.scalastyle.org/rules-0.1.0.html
> > >
> > > Cheers
> > >
> > > On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
> > nicholas.chammas@gmail.com
> > >> wrote:
> > >
> > >> As discussed here <https://github.com/apache/spark/pull/2619>, it
> > would be
> > >> good to extend our Scala style checks to programmatically enforce as
> > many
> > >> of our style rules as possible.
> > >>
> > >> Does anyone know if it's relatively straightforward to enforce
> > additional
> > >> rules like the "no trailing spaces" rule mentioned in the linked PR?
> > >>
> > >> Nick
> > >>
> >
>
Re: Extending Scala style checks
Posted by Nicholas Chammas <ni...@gmail.com>.
Ah, since there appears to be a built-in rule for end-of-line whitespace,
Michael and Cheng, y'all should be able to add this in pretty easily.
Nick
On Wed, Oct 1, 2014 at 6:37 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey Nick,
>
> We can always take built-in rules. Back when we added this Prashant
> Sharma actually did some great work that lets us write our own style
> rules in cases where rules don't exist.
>
> You can see some existing rules here:
>
> https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
>
> Prashant has over time contributed a lot of our custom rules upstream
> to stalastyle, so now there are only a couple there.
>
> - Patrick
>
> On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
> > Please take a look at WhitespaceEndOfLineChecker under:
> > http://www.scalastyle.org/rules-0.1.0.html
> >
> > Cheers
> >
> > On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com
> >> wrote:
> >
> >> As discussed here <https://github.com/apache/spark/pull/2619>, it
> would be
> >> good to extend our Scala style checks to programmatically enforce as
> many
> >> of our style rules as possible.
> >>
> >> Does anyone know if it's relatively straightforward to enforce
> additional
> >> rules like the "no trailing spaces" rule mentioned in the linked PR?
> >>
> >> Nick
> >>
>
Re: Extending Scala style checks
Posted by Patrick Wendell <pw...@gmail.com>.
Hey Nick,
We can always take built-in rules. Back when we added this Prashant
Sharma actually did some great work that lets us write our own style
rules in cases where rules don't exist.
You can see some existing rules here:
https://github.com/apache/spark/tree/master/project/spark-style/src/main/scala/org/apache/spark/scalastyle
Prashant has over time contributed a lot of our custom rules upstream
to stalastyle, so now there are only a couple there.
- Patrick
On Wed, Oct 1, 2014 at 2:36 PM, Ted Yu <yu...@gmail.com> wrote:
> Please take a look at WhitespaceEndOfLineChecker under:
> http://www.scalastyle.org/rules-0.1.0.html
>
> Cheers
>
> On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <nicholas.chammas@gmail.com
>> wrote:
>
>> As discussed here <https://github.com/apache/spark/pull/2619>, it would be
>> good to extend our Scala style checks to programmatically enforce as many
>> of our style rules as possible.
>>
>> Does anyone know if it's relatively straightforward to enforce additional
>> rules like the "no trailing spaces" rule mentioned in the linked PR?
>>
>> Nick
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Extending Scala style checks
Posted by Ted Yu <yu...@gmail.com>.
Please take a look at WhitespaceEndOfLineChecker under:
http://www.scalastyle.org/rules-0.1.0.html
Cheers
On Wed, Oct 1, 2014 at 2:01 PM, Nicholas Chammas <nicholas.chammas@gmail.com
> wrote:
> As discussed here <https://github.com/apache/spark/pull/2619>, it would be
> good to extend our Scala style checks to programmatically enforce as many
> of our style rules as possible.
>
> Does anyone know if it's relatively straightforward to enforce additional
> rules like the "no trailing spaces" rule mentioned in the linked PR?
>
> Nick
>