Making Policy decisions on artifacts and how they look.

[Davanum Srinivas <da...@gmail.com>]

Dear Infra folks,

Looks like we have some folks in the repository@ mailing list who are
making ASF wide policy decisions without consulting anyone. Not this
list, not the members list, not the pmc's list, nor the board.

WDYT?

thanks,
dims

Re: Making Policy decisions on artifacts and how they look.

[Davanum Srinivas <da...@gmail.com>]

Steve,

Am asking them to publish the guidelines and a date for conforming to
it. What's wrong with that?

-- dims

On 12/12/06, Steve Loughran <st...@gmail.com> wrote:
> On 12/12/06, Davanum Srinivas <da...@gmail.com> wrote:
> > Dear Infra folks,
> >
> > Looks like we have some folks in the repository@ mailing list who are
> > making ASF wide policy decisions without consulting anyone. Not this
> > list, not the members list, not the pmc's list, nor the board.
> >
>
> I dont see anyone making ASF-wide policy decisions.
>
> I see people trying to stay in control of the Maven repository -a repo
> for a single project- and struggling to stay in control against the
> appearance of random artifacts which often come from apache projects
> without adequate signoff from the PMCs and with pretty low quality
> metadata. The fact that other projects (ivy, me) use the same repo is
> because there is lots of stuff in there, rather than because its the
> official 'apache' repo.
>
> Nobody is telling you what to call the project; you can have anything
> except slash, ASCII<32, space or < >. Carlos is merely trying to
> impose a better hierarchy on how things are stored *in the
> repository*. The M1 repo had a flat naming system which puts too much
> load on the root directory. A hierarchy is better. If everything
> WS-related was under org.apache.ws then you could browse to
> org/apache/ws and see the entire suite of artifacts. If we go flat
> under org.apache then it ends up having the scale problem.
>
> There's more control of external artifacts where the files get pulled
> in by hand; in that context things are locked down and less messy.
> Maybe we should do that for apache stuff too, instead of being so
> trusting.
>
> Sorry,
>
> -Steve
>


-- 
Davanum Srinivas : http://www.wso2.net (Oxygen for Web Service Developers)

Re: Making Policy decisions on artifacts and how they look.

[Steve Loughran <st...@gmail.com>]

On 12/12/06, Davanum Srinivas <da...@gmail.com> wrote:
> Dear Infra folks,
>
> Looks like we have some folks in the repository@ mailing list who are
> making ASF wide policy decisions without consulting anyone. Not this
> list, not the members list, not the pmc's list, nor the board.
>

I dont see anyone making ASF-wide policy decisions.

I see people trying to stay in control of the Maven repository -a repo
for a single project- and struggling to stay in control against the
appearance of random artifacts which often come from apache projects
without adequate signoff from the PMCs and with pretty low quality
metadata. The fact that other projects (ivy, me) use the same repo is
because there is lots of stuff in there, rather than because its the
official 'apache' repo.

Nobody is telling you what to call the project; you can have anything
except slash, ASCII<32, space or < >. Carlos is merely trying to
impose a better hierarchy on how things are stored *in the
repository*. The M1 repo had a flat naming system which puts too much
load on the root directory. A hierarchy is better. If everything
WS-related was under org.apache.ws then you could browse to
org/apache/ws and see the entire suite of artifacts. If we go flat
under org.apache then it ends up having the scale problem.

There's more control of external artifacts where the files get pulled
in by hand; in that context things are locked down and less messy.
Maybe we should do that for apache stuff too, instead of being so
trusting.

Sorry,

-Steve

Re: Making Policy decisions on artifacts and how they look.

[Brett Porter <br...@apache.org>]

Joshua,

I think we'd all agree with these concerns, but the aim of  
'repository-land' is to try and curb both of those behaviours.

- Brett

On 13/12/2006, at 8:28 AM, Joshua Slive wrote:

> On 12/12/06, Steve Loughran <st...@gmail.com> wrote:
>> On 12/12/06, Joshua Slive <jo...@slive.ca> wrote:
>>
>> >
>> > Having said that, I've always been rather freaked out by the  
>> goings-on
>> > in repository-land.
>>
>> How so?
>
> 1. Total lack of control on what goes in and out.
>
> 2. Tying project builds to existence of specific internet resources --
> and even worse if they are apache.org resources.
>
> But I admit that I don't follow any of it very closely.  What mainly
> gets me is when some change on some server someplace causes people to
> come screaming "you broke my build".  Having changes on someone else's
> server break builds of software on your computer just seems like way
> too fragile of an ecosystem to me.
>
> Joshua.

Re: Making Policy decisions on artifacts and how they look.

[Carlos Sanchez <ca...@apache.org>]

On 12/13/06, Steve Loughran <st...@gmail.com> wrote:
> In the m1/m2 repo, we declare at compile time what versions of things
> we want. If I want to run against the commons-logging 1.1 jar I say
> so, and there is no way to push a later version at my app. We can't
> force commons-logging to release a 1.1.1 just to fix the metadata, yet
> its the only means we have today to fix the problem.

there is a way, you can always override a dependency version. Maven
repo provides the version used to build it, but you can just change
it. Projects can release also the same binary with different
dependency sets if they want to help users.

-- 
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
                             -- The Princess Bride

Re: Making Policy decisions on artifacts and how they look.

[Steve Loughran <st...@gmail.com>]

On 13/12/06, Carlos Sanchez <ca...@apache.org> wrote:
> On 12/12/06, Steve Loughran <st...@gmail.com> wrote:
> > Carlos, Wendy and others are actually so rigorous about not breaking
> > peoples builds they prefer to leave artifacts in the wrong place and
> > with bad metadata (eg. commons-logging 1.1.) rather than correct
> > obvious defects. That's why we need to lock down the transition of
> > artifacts from staging to live. IMO part of the problem is in the
> > clients which never invalidate their cache; they assume metadata is
> > always complete and perfect. This makes it hard to correct metadata
> > bugs because the changes only propagate to new machines.
>
> I think it's a matter of getting people used to it. Nobody thinks
> possible that an RPM changes it's dependencies, a new version is
> released. In the repo is the same case, just that some projects still
> don't care about it and don't help too much.

There's a couple of diffs with RPM/deb files. The main one is that
they are normally released by redistributors who have their own number
scheme

My desktops subscribe to their apt repositories and get
firefox-1.5.06-ubuntu-29.deb
my server subscribes to the Suse repo and gets firefox-1.5.06-suse-8.rpm.

The numbering is unumportant to me, as long as it is consistent across
all artifacts in the different repositories. It is not ever included
in the makefiles to build things, as the versions of dependencies and
the means to build them are not defined at this point --there's
nothing that does the retrieval. You have to have the dependencies in
place as a prerequisite (exception. apt-get from source, which does
the download, build and install based on the metadata. even then, its
not the makefiles that care)

In the m1/m2 repo, we declare at compile time what versions of things
we want. If I want to run against the commons-logging 1.1 jar I say
so, and there is no way to push a later version at my app. We can't
force commons-logging to release a 1.1.1 just to fix the metadata, yet
its the only means we have today to fix the problem.

-steve

Re: Making Policy decisions on artifacts and how they look.

[Carlos Sanchez <ca...@apache.org>]

On 12/12/06, Steve Loughran <st...@gmail.com> wrote:
> Carlos, Wendy and others are actually so rigorous about not breaking
> peoples builds they prefer to leave artifacts in the wrong place and
> with bad metadata (eg. commons-logging 1.1.) rather than correct
> obvious defects. That's why we need to lock down the transition of
> artifacts from staging to live. IMO part of the problem is in the
> clients which never invalidate their cache; they assume metadata is
> always complete and perfect. This makes it hard to correct metadata
> bugs because the changes only propagate to new machines.

I think it's a matter of getting people used to it. Nobody thinks
possible that an RPM changes it's dependencies, a new version is
released. In the repo is the same case, just that some projects still
don't care about it and don't help too much.

Re: Making Policy decisions on artifacts and how they look.

[Steve Loughran <st...@gmail.com>]

On 12/12/06, Joshua Slive <jo...@slive.ca> wrote:
> On 12/12/06, Steve Loughran <st...@gmail.com> wrote:
> > On 12/12/06, Joshua Slive <jo...@slive.ca> wrote:
> >
> > >
> > > Having said that, I've always been rather freaked out by the goings-on
> > > in repository-land.
> >
> > How so?
>
> 1. Total lack of control on what goes in and out.
>

+1

I'd really like to get log data from one of the main repos, to see
which artifacts are popular; and which are being looked for and not
found. Is there any way to do this?

> 2. Tying project builds to existence of specific internet resources --
> and even worse if they are apache.org resources.

oh, that reminds me. For the rules. No -SNAPSHOT dependencies.

>
> But I admit that I don't follow any of it very closely.  What mainly
> gets me is when some change on some server someplace causes people to
> come screaming "you broke my build".  Having changes on someone else's
> server break builds of software on your computer just seems like way
> too fragile of an ecosystem to me.
>

We call it a distributed system :)

When I build up a new linux distro I add various 'unofficial' apt
repos to the world; I dont live in that much fear that the repo that
serves Xine goes away, or that the dependency logic is broken. Why
should things be different in the Java space?

Carlos, Wendy and others are actually so rigorous about not breaking
peoples builds they prefer to leave artifacts in the wrong place and
with bad metadata (eg. commons-logging 1.1.) rather than correct
obvious defects. That's why we need to lock down the transition of
artifacts from staging to live. IMO part of the problem is in the
clients which never invalidate their cache; they assume metadata is
always complete and perfect. This makes it hard to correct metadata
bugs because the changes only propagate to new machines.

-Steve

Re: Making Policy decisions on artifacts and how they look.

[Joshua Slive <jo...@slive.ca>]

On 12/12/06, Steve Loughran <st...@gmail.com> wrote:
> On 12/12/06, Joshua Slive <jo...@slive.ca> wrote:
>
> >
> > Having said that, I've always been rather freaked out by the goings-on
> > in repository-land.
>
> How so?

1. Total lack of control on what goes in and out.

2. Tying project builds to existence of specific internet resources --
and even worse if they are apache.org resources.

But I admit that I don't follow any of it very closely.  What mainly
gets me is when some change on some server someplace causes people to
come screaming "you broke my build".  Having changes on someone else's
server break builds of software on your computer just seems like way
too fragile of an ecosystem to me.

Joshua.

Re: Making Policy decisions on artifacts and how they look.

[Steve Loughran <st...@gmail.com>]

On 12/12/06, Joshua Slive <jo...@slive.ca> wrote:

>
> Having said that, I've always been rather freaked out by the goings-on
> in repository-land.

How so?

Re: Making Policy decisions on artifacts and how they look.

[Joshua Slive <jo...@slive.ca>]

On 12/12/06, Davanum Srinivas <da...@gmail.com> wrote:
> Dear Infra folks,
>
> Looks like we have some folks in the repository@ mailing list who are
> making ASF wide policy decisions without consulting anyone. Not this
> list, not the members list, not the pmc's list, nor the board.
>
> WDYT?

Infrastructure projects run by consensus, the same as the rest of the
ASF.  If you think they are doing something wrong, then you need to
convince them of that.  Asking the board to get involved in operations
is really a last resort for a failed community.

Having said that, I've always been rather freaked out by the goings-on
in repository-land.  I just don't have the bandwidth to keep up with
it.  One choice would be to ask them to come back to infrastructure@
for everything other than day-to-day stuff.

Joshua.


Joshua.