Should we continue to maintain fine-grained python BUILD targets?

[Bill Farner <wf...@apache.org>]

We currently have ~111 BUILD targets serving our python code, and ~106 test
targets:

$ find . -name BUILD | xargs grep python_library | wc -l
111

$ find . -name BUILD | xargs grep python_test | wc -l
106

This presents a non-trivial maintenance burden, and i'm not convinced the
benefits are worth it.  Now that we have a good IDE story, refactors are
painless in the source code, but painful when you need to hand-edit all
affected BUILD targets.

I can think of several motivations for using fine-grained targets
1. Running specific unit tests with pants
2. Limiting the sources included in a binary
3. Carving out libraries to be published

I'm not convinced these benefits outweigh the maintenance burden of these
files.  I propose we reduce our BUILD targets to ~equal to the number of
binaries we produce, possibly with a few for libraries shared between
components.

Does anyone else have opinions on this?  Is there something i'm overlooking?


-=Bill

Re: Should we continue to maintain fine-grained python BUILD targets?

[Joseph Smith <ya...@gmail.com>]

> On Jan 7, 2015, at 11:24 AM, Kevin Sweeney <ks...@twitter.com.INVALID> wrote:
> 
> +1 to Josh's statement - if we're going to stay with pants we should follow
> the 1 directory, 1 package, 1 BUILD file convention (though if we stay the
> course there I suggest we write a tool to generate BUILD files based on
> static analysis of import statements).
> 
> +1 to exploring tox+setup.py and replacing pants completely (or at least
> for everything but thrift codegen).
> 
> To Joe's point: we can still select individual tests with the -k flag to
> py.test/tox (that's pretty much what happens when you right-click > run
> test in pycharm), so we don't lose the ability to run tests granularly in
> either scenario.

Aha, good point, thanks for the hint :)


> 
> On Wed, Jan 7, 2015 at 11:09 AM, John Sirois <jo...@gmail.com> wrote:
> 
>> On Wed, Jan 7, 2015 at 12:05 PM, Joseph Smith <ya...@gmail.com> wrote:
>> 
>>> I found it ~annoying to test the client in a few cases where multiple
>>> targets were in one large test target- I’m in favor of fine-grained BUILD
>>> targets despite the (what I consider slight) additional overhead.
>>> 
>> 
>> NB though that you can use pytest's -k to select individual tests no matter
>> how many files or targets are passed under tox and pants respectively, so I
>> think this particular argument for or against washes out.
>> 
>> 
>>>> On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jc...@twopensource.com>
>>> wrote:
>>>> 
>>>> I agree, pants is heavily biased towards 1 directory, 1 package, 1
>> build
>>>> target. If we're going to stay with pants I'd prefer to stick with that
>>>> convention. If we're going to simplify the build structure I think we
>>>> should take that as impetus to move away from pants entirely.
>>>> 
>>>> On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org>
>>> wrote:
>>>> 
>>>>> 4) Enforcing sensible software hygiene.
>>>>> 
>>>>> The pants project itself for the longest time had a single root rglobs
>>>>> python_library target.  I spent a few days splitting up that project
>>> into
>>>>> fine-grained targets and it uncovered a number of cyclical
>> dependencies
>>> and
>>>>> abstraction leaks that simply would've been impossible to make with
>>> better
>>>>> target encapsulation.  If everyone is an expert in the project, BUILD
>>>>> management can be tedious overhead, but for novice contributors,
>>>>> fine-grained BUILDs can be an effective linting mechanism to prevent
>>> poor
>>>>> code structure and advise the design process.
>>>>> 
>>>>> This is entirely a trade-off.  I'm not sure you could get the same
>>>>> protections with coarser granularity than what we have now.  That
>> being
>>>>> said, if we move forward with simplification, I'd propose we go all-in
>>> and
>>>>> remove pants entirely in favor of setup.py and tox.
>>>>> 
>>>>> ~brian
>>>>> 
>>>>> 
>>>>> 
>>>>> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org>
>> wrote:
>>>>> 
>>>>>> We currently have ~111 BUILD targets serving our python code, and
>> ~106
>>>>> test
>>>>>> targets:
>>>>>> 
>>>>>> $ find . -name BUILD | xargs grep python_library | wc -l
>>>>>> 111
>>>>>> 
>>>>>> $ find . -name BUILD | xargs grep python_test | wc -l
>>>>>> 106
>>>>>> 
>>>>>> This presents a non-trivial maintenance burden, and i'm not convinced
>>> the
>>>>>> benefits are worth it.  Now that we have a good IDE story, refactors
>>> are
>>>>>> painless in the source code, but painful when you need to hand-edit
>> all
>>>>>> affected BUILD targets.
>>>>>> 
>>>>>> I can think of several motivations for using fine-grained targets
>>>>>> 1. Running specific unit tests with pants
>>>>>> 2. Limiting the sources included in a binary
>>>>>> 3. Carving out libraries to be published
>>>>>> 
>>>>>> I'm not convinced these benefits outweigh the maintenance burden of
>>> these
>>>>>> files.  I propose we reduce our BUILD targets to ~equal to the number
>>> of
>>>>>> binaries we produce, possibly with a few for libraries shared between
>>>>>> components.
>>>>>> 
>>>>>> Does anyone else have opinions on this?  Is there something i'm
>>>>>> overlooking?
>>>>>> 
>>>>>> 
>>>>>> -=Bill
>>>>>> 
>>>>> 
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Kevin Sweeney
> @kts

Re: Should we continue to maintain fine-grained python BUILD targets?

[Kevin Sweeney <ks...@twitter.com.INVALID>]

+1 to Josh's statement - if we're going to stay with pants we should follow
the 1 directory, 1 package, 1 BUILD file convention (though if we stay the
course there I suggest we write a tool to generate BUILD files based on
static analysis of import statements).

+1 to exploring tox+setup.py and replacing pants completely (or at least
for everything but thrift codegen).

To Joe's point: we can still select individual tests with the -k flag to
py.test/tox (that's pretty much what happens when you right-click > run
test in pycharm), so we don't lose the ability to run tests granularly in
either scenario.

On Wed, Jan 7, 2015 at 11:09 AM, John Sirois <jo...@gmail.com> wrote:

> On Wed, Jan 7, 2015 at 12:05 PM, Joseph Smith <ya...@gmail.com> wrote:
>
> > I found it ~annoying to test the client in a few cases where multiple
> > targets were in one large test target- I’m in favor of fine-grained BUILD
> > targets despite the (what I consider slight) additional overhead.
> >
>
> NB though that you can use pytest's -k to select individual tests no matter
> how many files or targets are passed under tox and pants respectively, so I
> think this particular argument for or against washes out.
>
>
> > > On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jc...@twopensource.com>
> > wrote:
> > >
> > > I agree, pants is heavily biased towards 1 directory, 1 package, 1
> build
> > > target. If we're going to stay with pants I'd prefer to stick with that
> > > convention. If we're going to simplify the build structure I think we
> > > should take that as impetus to move away from pants entirely.
> > >
> > > On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org>
> > wrote:
> > >
> > >> 4) Enforcing sensible software hygiene.
> > >>
> > >> The pants project itself for the longest time had a single root rglobs
> > >> python_library target.  I spent a few days splitting up that project
> > into
> > >> fine-grained targets and it uncovered a number of cyclical
> dependencies
> > and
> > >> abstraction leaks that simply would've been impossible to make with
> > better
> > >> target encapsulation.  If everyone is an expert in the project, BUILD
> > >> management can be tedious overhead, but for novice contributors,
> > >> fine-grained BUILDs can be an effective linting mechanism to prevent
> > poor
> > >> code structure and advise the design process.
> > >>
> > >> This is entirely a trade-off.  I'm not sure you could get the same
> > >> protections with coarser granularity than what we have now.  That
> being
> > >> said, if we move forward with simplification, I'd propose we go all-in
> > and
> > >> remove pants entirely in favor of setup.py and tox.
> > >>
> > >> ~brian
> > >>
> > >>
> > >>
> > >> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org>
> wrote:
> > >>
> > >>> We currently have ~111 BUILD targets serving our python code, and
> ~106
> > >> test
> > >>> targets:
> > >>>
> > >>> $ find . -name BUILD | xargs grep python_library | wc -l
> > >>> 111
> > >>>
> > >>> $ find . -name BUILD | xargs grep python_test | wc -l
> > >>> 106
> > >>>
> > >>> This presents a non-trivial maintenance burden, and i'm not convinced
> > the
> > >>> benefits are worth it.  Now that we have a good IDE story, refactors
> > are
> > >>> painless in the source code, but painful when you need to hand-edit
> all
> > >>> affected BUILD targets.
> > >>>
> > >>> I can think of several motivations for using fine-grained targets
> > >>> 1. Running specific unit tests with pants
> > >>> 2. Limiting the sources included in a binary
> > >>> 3. Carving out libraries to be published
> > >>>
> > >>> I'm not convinced these benefits outweigh the maintenance burden of
> > these
> > >>> files.  I propose we reduce our BUILD targets to ~equal to the number
> > of
> > >>> binaries we produce, possibly with a few for libraries shared between
> > >>> components.
> > >>>
> > >>> Does anyone else have opinions on this?  Is there something i'm
> > >>> overlooking?
> > >>>
> > >>>
> > >>> -=Bill
> > >>>
> > >>
> >
> >
>



-- 
Kevin Sweeney
@kts

Re: Should we continue to maintain fine-grained python BUILD targets?

[Kevin Sweeney <ke...@apache.org>]

+1 to Josh's statement - if we're going to stay with pants we should follow
the 1 directory, 1 package, 1 BUILD file convention (though if we stay the
course there I suggest we write a tool to generate BUILD files based on
static analysis of import statements).

+1 to exploring tox+setup.py and replacing pants completely (or at least
for everything but thrift codegen).

To Joe's point: we can still select individual tests with the -k flag to
py.test/tox (that's pretty much what happens when you right-click > run
test in pycharm), so we don't lose the ability to run tests granularly in
either scenario.

On Wed, Jan 7, 2015 at 11:09 AM, John Sirois <jo...@gmail.com> wrote:

> On Wed, Jan 7, 2015 at 12:05 PM, Joseph Smith <ya...@gmail.com> wrote:
>
> > I found it ~annoying to test the client in a few cases where multiple
> > targets were in one large test target- I’m in favor of fine-grained BUILD
> > targets despite the (what I consider slight) additional overhead.
> >
>
> NB though that you can use pytest's -k to select individual tests no matter
> how many files or targets are passed under tox and pants respectively, so I
> think this particular argument for or against washes out.
>
>
> > > On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jc...@twopensource.com>
> > wrote:
> > >
> > > I agree, pants is heavily biased towards 1 directory, 1 package, 1
> build
> > > target. If we're going to stay with pants I'd prefer to stick with that
> > > convention. If we're going to simplify the build structure I think we
> > > should take that as impetus to move away from pants entirely.
> > >
> > > On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org>
> > wrote:
> > >
> > >> 4) Enforcing sensible software hygiene.
> > >>
> > >> The pants project itself for the longest time had a single root rglobs
> > >> python_library target.  I spent a few days splitting up that project
> > into
> > >> fine-grained targets and it uncovered a number of cyclical
> dependencies
> > and
> > >> abstraction leaks that simply would've been impossible to make with
> > better
> > >> target encapsulation.  If everyone is an expert in the project, BUILD
> > >> management can be tedious overhead, but for novice contributors,
> > >> fine-grained BUILDs can be an effective linting mechanism to prevent
> > poor
> > >> code structure and advise the design process.
> > >>
> > >> This is entirely a trade-off.  I'm not sure you could get the same
> > >> protections with coarser granularity than what we have now.  That
> being
> > >> said, if we move forward with simplification, I'd propose we go all-in
> > and
> > >> remove pants entirely in favor of setup.py and tox.
> > >>
> > >> ~brian
> > >>
> > >>
> > >>
> > >> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org>
> wrote:
> > >>
> > >>> We currently have ~111 BUILD targets serving our python code, and
> ~106
> > >> test
> > >>> targets:
> > >>>
> > >>> $ find . -name BUILD | xargs grep python_library | wc -l
> > >>> 111
> > >>>
> > >>> $ find . -name BUILD | xargs grep python_test | wc -l
> > >>> 106
> > >>>
> > >>> This presents a non-trivial maintenance burden, and i'm not convinced
> > the
> > >>> benefits are worth it.  Now that we have a good IDE story, refactors
> > are
> > >>> painless in the source code, but painful when you need to hand-edit
> all
> > >>> affected BUILD targets.
> > >>>
> > >>> I can think of several motivations for using fine-grained targets
> > >>> 1. Running specific unit tests with pants
> > >>> 2. Limiting the sources included in a binary
> > >>> 3. Carving out libraries to be published
> > >>>
> > >>> I'm not convinced these benefits outweigh the maintenance burden of
> > these
> > >>> files.  I propose we reduce our BUILD targets to ~equal to the number
> > of
> > >>> binaries we produce, possibly with a few for libraries shared between
> > >>> components.
> > >>>
> > >>> Does anyone else have opinions on this?  Is there something i'm
> > >>> overlooking?
> > >>>
> > >>>
> > >>> -=Bill
> > >>>
> > >>
> >
> >
>

Re: Should we continue to maintain fine-grained python BUILD targets?

[John Sirois <jo...@gmail.com>]

On Wed, Jan 7, 2015 at 12:05 PM, Joseph Smith <ya...@gmail.com> wrote:

> I found it ~annoying to test the client in a few cases where multiple
> targets were in one large test target- I’m in favor of fine-grained BUILD
> targets despite the (what I consider slight) additional overhead.
>

NB though that you can use pytest's -k to select individual tests no matter
how many files or targets are passed under tox and pants respectively, so I
think this particular argument for or against washes out.


> > On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jc...@twopensource.com>
> wrote:
> >
> > I agree, pants is heavily biased towards 1 directory, 1 package, 1 build
> > target. If we're going to stay with pants I'd prefer to stick with that
> > convention. If we're going to simplify the build structure I think we
> > should take that as impetus to move away from pants entirely.
> >
> > On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org>
> wrote:
> >
> >> 4) Enforcing sensible software hygiene.
> >>
> >> The pants project itself for the longest time had a single root rglobs
> >> python_library target.  I spent a few days splitting up that project
> into
> >> fine-grained targets and it uncovered a number of cyclical dependencies
> and
> >> abstraction leaks that simply would've been impossible to make with
> better
> >> target encapsulation.  If everyone is an expert in the project, BUILD
> >> management can be tedious overhead, but for novice contributors,
> >> fine-grained BUILDs can be an effective linting mechanism to prevent
> poor
> >> code structure and advise the design process.
> >>
> >> This is entirely a trade-off.  I'm not sure you could get the same
> >> protections with coarser granularity than what we have now.  That being
> >> said, if we move forward with simplification, I'd propose we go all-in
> and
> >> remove pants entirely in favor of setup.py and tox.
> >>
> >> ~brian
> >>
> >>
> >>
> >> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org> wrote:
> >>
> >>> We currently have ~111 BUILD targets serving our python code, and ~106
> >> test
> >>> targets:
> >>>
> >>> $ find . -name BUILD | xargs grep python_library | wc -l
> >>> 111
> >>>
> >>> $ find . -name BUILD | xargs grep python_test | wc -l
> >>> 106
> >>>
> >>> This presents a non-trivial maintenance burden, and i'm not convinced
> the
> >>> benefits are worth it.  Now that we have a good IDE story, refactors
> are
> >>> painless in the source code, but painful when you need to hand-edit all
> >>> affected BUILD targets.
> >>>
> >>> I can think of several motivations for using fine-grained targets
> >>> 1. Running specific unit tests with pants
> >>> 2. Limiting the sources included in a binary
> >>> 3. Carving out libraries to be published
> >>>
> >>> I'm not convinced these benefits outweigh the maintenance burden of
> these
> >>> files.  I propose we reduce our BUILD targets to ~equal to the number
> of
> >>> binaries we produce, possibly with a few for libraries shared between
> >>> components.
> >>>
> >>> Does anyone else have opinions on this?  Is there something i'm
> >>> overlooking?
> >>>
> >>>
> >>> -=Bill
> >>>
> >>
>
>

Re: Should we continue to maintain fine-grained python BUILD targets?

[Joseph Smith <ya...@gmail.com>]

I found it ~annoying to test the client in a few cases where multiple targets were in one large test target- I’m in favor of fine-grained BUILD targets despite the (what I consider slight) additional overhead.

> On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jc...@twopensource.com> wrote:
> 
> I agree, pants is heavily biased towards 1 directory, 1 package, 1 build
> target. If we're going to stay with pants I'd prefer to stick with that
> convention. If we're going to simplify the build structure I think we
> should take that as impetus to move away from pants entirely.
> 
> On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org> wrote:
> 
>> 4) Enforcing sensible software hygiene.
>> 
>> The pants project itself for the longest time had a single root rglobs
>> python_library target.  I spent a few days splitting up that project into
>> fine-grained targets and it uncovered a number of cyclical dependencies and
>> abstraction leaks that simply would've been impossible to make with better
>> target encapsulation.  If everyone is an expert in the project, BUILD
>> management can be tedious overhead, but for novice contributors,
>> fine-grained BUILDs can be an effective linting mechanism to prevent poor
>> code structure and advise the design process.
>> 
>> This is entirely a trade-off.  I'm not sure you could get the same
>> protections with coarser granularity than what we have now.  That being
>> said, if we move forward with simplification, I'd propose we go all-in and
>> remove pants entirely in favor of setup.py and tox.
>> 
>> ~brian
>> 
>> 
>> 
>> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org> wrote:
>> 
>>> We currently have ~111 BUILD targets serving our python code, and ~106
>> test
>>> targets:
>>> 
>>> $ find . -name BUILD | xargs grep python_library | wc -l
>>> 111
>>> 
>>> $ find . -name BUILD | xargs grep python_test | wc -l
>>> 106
>>> 
>>> This presents a non-trivial maintenance burden, and i'm not convinced the
>>> benefits are worth it.  Now that we have a good IDE story, refactors are
>>> painless in the source code, but painful when you need to hand-edit all
>>> affected BUILD targets.
>>> 
>>> I can think of several motivations for using fine-grained targets
>>> 1. Running specific unit tests with pants
>>> 2. Limiting the sources included in a binary
>>> 3. Carving out libraries to be published
>>> 
>>> I'm not convinced these benefits outweigh the maintenance burden of these
>>> files.  I propose we reduce our BUILD targets to ~equal to the number of
>>> binaries we produce, possibly with a few for libraries shared between
>>> components.
>>> 
>>> Does anyone else have opinions on this?  Is there something i'm
>>> overlooking?
>>> 
>>> 
>>> -=Bill
>>> 
>> 

Re: Should we continue to maintain fine-grained python BUILD targets?

[Joshua Cohen <jc...@twopensource.com>]

I agree, pants is heavily biased towards 1 directory, 1 package, 1 build
target. If we're going to stay with pants I'd prefer to stick with that
convention. If we're going to simplify the build structure I think we
should take that as impetus to move away from pants entirely.

On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org> wrote:

> 4) Enforcing sensible software hygiene.
>
> The pants project itself for the longest time had a single root rglobs
> python_library target.  I spent a few days splitting up that project into
> fine-grained targets and it uncovered a number of cyclical dependencies and
> abstraction leaks that simply would've been impossible to make with better
> target encapsulation.  If everyone is an expert in the project, BUILD
> management can be tedious overhead, but for novice contributors,
> fine-grained BUILDs can be an effective linting mechanism to prevent poor
> code structure and advise the design process.
>
> This is entirely a trade-off.  I'm not sure you could get the same
> protections with coarser granularity than what we have now.  That being
> said, if we move forward with simplification, I'd propose we go all-in and
> remove pants entirely in favor of setup.py and tox.
>
> ~brian
>
>
>
> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org> wrote:
>
> > We currently have ~111 BUILD targets serving our python code, and ~106
> test
> > targets:
> >
> > $ find . -name BUILD | xargs grep python_library | wc -l
> > 111
> >
> > $ find . -name BUILD | xargs grep python_test | wc -l
> > 106
> >
> > This presents a non-trivial maintenance burden, and i'm not convinced the
> > benefits are worth it.  Now that we have a good IDE story, refactors are
> > painless in the source code, but painful when you need to hand-edit all
> > affected BUILD targets.
> >
> > I can think of several motivations for using fine-grained targets
> > 1. Running specific unit tests with pants
> > 2. Limiting the sources included in a binary
> > 3. Carving out libraries to be published
> >
> > I'm not convinced these benefits outweigh the maintenance burden of these
> > files.  I propose we reduce our BUILD targets to ~equal to the number of
> > binaries we produce, possibly with a few for libraries shared between
> > components.
> >
> > Does anyone else have opinions on this?  Is there something i'm
> > overlooking?
> >
> >
> > -=Bill
> >
>

Re: Should we continue to maintain fine-grained python BUILD targets?

[Zameer Manji <zm...@twopensource.com>]

If we decide to simplify the targets then I'm also in favor of going all in
and removing pants in favour of setup.py and tox.

On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wi...@apache.org> wrote:

> 4) Enforcing sensible software hygiene.
>
> The pants project itself for the longest time had a single root rglobs
> python_library target.  I spent a few days splitting up that project into
> fine-grained targets and it uncovered a number of cyclical dependencies and
> abstraction leaks that simply would've been impossible to make with better
> target encapsulation.  If everyone is an expert in the project, BUILD
> management can be tedious overhead, but for novice contributors,
> fine-grained BUILDs can be an effective linting mechanism to prevent poor
> code structure and advise the design process.
>
> This is entirely a trade-off.  I'm not sure you could get the same
> protections with coarser granularity than what we have now.  That being
> said, if we move forward with simplification, I'd propose we go all-in and
> remove pants entirely in favor of setup.py and tox.
>
> ~brian
>
>
>
> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org> wrote:
>
> > We currently have ~111 BUILD targets serving our python code, and ~106
> test
> > targets:
> >
> > $ find . -name BUILD | xargs grep python_library | wc -l
> > 111
> >
> > $ find . -name BUILD | xargs grep python_test | wc -l
> > 106
> >
> > This presents a non-trivial maintenance burden, and i'm not convinced the
> > benefits are worth it.  Now that we have a good IDE story, refactors are
> > painless in the source code, but painful when you need to hand-edit all
> > affected BUILD targets.
> >
> > I can think of several motivations for using fine-grained targets
> > 1. Running specific unit tests with pants
> > 2. Limiting the sources included in a binary
> > 3. Carving out libraries to be published
> >
> > I'm not convinced these benefits outweigh the maintenance burden of these
> > files.  I propose we reduce our BUILD targets to ~equal to the number of
> > binaries we produce, possibly with a few for libraries shared between
> > components.
> >
> > Does anyone else have opinions on this?  Is there something i'm
> > overlooking?
> >
> >
> > -=Bill
> >
>



-- 
Zameer Manji

Re: Should we continue to maintain fine-grained python BUILD targets?

[Brian Wickman <wi...@apache.org>]

4) Enforcing sensible software hygiene.

The pants project itself for the longest time had a single root rglobs
python_library target.  I spent a few days splitting up that project into
fine-grained targets and it uncovered a number of cyclical dependencies and
abstraction leaks that simply would've been impossible to make with better
target encapsulation.  If everyone is an expert in the project, BUILD
management can be tedious overhead, but for novice contributors,
fine-grained BUILDs can be an effective linting mechanism to prevent poor
code structure and advise the design process.

This is entirely a trade-off.  I'm not sure you could get the same
protections with coarser granularity than what we have now.  That being
said, if we move forward with simplification, I'd propose we go all-in and
remove pants entirely in favor of setup.py and tox.

~brian



On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wf...@apache.org> wrote:

> We currently have ~111 BUILD targets serving our python code, and ~106 test
> targets:
>
> $ find . -name BUILD | xargs grep python_library | wc -l
> 111
>
> $ find . -name BUILD | xargs grep python_test | wc -l
> 106
>
> This presents a non-trivial maintenance burden, and i'm not convinced the
> benefits are worth it.  Now that we have a good IDE story, refactors are
> painless in the source code, but painful when you need to hand-edit all
> affected BUILD targets.
>
> I can think of several motivations for using fine-grained targets
> 1. Running specific unit tests with pants
> 2. Limiting the sources included in a binary
> 3. Carving out libraries to be published
>
> I'm not convinced these benefits outweigh the maintenance burden of these
> files.  I propose we reduce our BUILD targets to ~equal to the number of
> binaries we produce, possibly with a few for libraries shared between
> components.
>
> Does anyone else have opinions on this?  Is there something i'm
> overlooking?
>
>
> -=Bill
>