You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by Janardhan <ja...@apache.org> on 2021/05/24 13:33:00 UTC
[QUESTION] Does the formal voting applies to all the deliverables
from a commit?
Hi all,
TL;DR: Should we sign Python artifacts and vote for them too?
The artifacts built from a commit SHA are to be signed[1] and voted on
for distribution via official release channels[2] downloads.apache.org,
docker[3].
Some projects choose their policy for specific platforms (pypi.org),
such as Apache Spark publishes pypi package[4] with Infrastructure's
knowledge.
This said, we have released our build artifacts produced by
maven via `mvn deploy -P'distribution,rat'` for 2.0 release[5] which
does not include our python API.
Previously SystemML had released[6] the python distribution, along with
other src, bin files.
Note: Pypi releases, github tags and docker images are convenience packages
and are not needed to go through formal voting.
[1] https://www.apache.org/legal/release-policy.html#release-signing
[2] https://infra.apache.org/release-distribution.html#unreleased
[3] https://hub.docker.com/u/apache
[4] https://pypi.org/project/pyspark/
[5] https://downloads.apache.org/systemds/2.0.0/
[6] https://downloads.apache.org/systemds/1.2.0/
Thanks and regards,
Janardhan
Re: [QUESTION] Does the formal voting applies to all the deliverables
from a commit?
Posted by Janardhan <ja...@gmail.com>.
>
> First, we do package
> the Python API into the source release (see systemds-2.0.0-src), but not
> the binary release. So related RC votes cover the Python API too
Thanks for confirmation.
> Personally, I would prefer joint releases where we try to keep the
> deployed Python artifacts consistent with the released main artifacts,
> avoid patch releases of the Python API, so the vote covers both
+1
Best regards,
Janardhan
On Mon, May 24, 2021 at 8:41 PM Janardhan <ja...@gmail.com>
wrote:
> Hi Sebastian,
>
> > Apache have a official docker which you pointed out many times.
> > If you feel the need that we use that, feel free to start converting to
> it.
> > I did not have the time to change it, especially since what we have
> currently is working for our needs.
>
> Sorry, it felt like an imposition. This one is not about the docker but
> writing down all the release channels that are available to us for record.
>
> The docker feature is tracked at SYSTEMDS-2941.
>
> Thanks,
> Janardhan
>
> On Mon, May 24, 2021 at 7:34 PM Matthias Boehm <mb...@gmail.com> wrote:
>
>> thanks for moving this discussion from the PR [1] to the dev list.
>>
>> Let's start by talking about the current status. First, we do package
>> the Python API into the source release (see systemds-2.0.0-src), but not
>> the binary release. So related RC votes cover the Python API too. The
>> pypi deployment corresponds to the released artifacts, but we allow
>> patch updates of the Python API because we currently only release
>> major/minor versions but no patches. The need for such patch updates
>> originated from different maturity levels of the new Python API and the
>> rest of the system.
>>
>> Personally, I would prefer joint releases where we try to keep the
>> deployed Python artifacts consistent with the released main artifacts,
>> avoid patch releases of the Python API, so the vote covers both. Similar
>> to concerns on the PR, I would not include the Python API into the maven
>> build process.
>>
>> [1] https://github.com/apache/systemds/pull/1225
>>
>> Regards,
>> Matthias
>>
>> On 5/24/2021 3:33 PM, Janardhan wrote:
>> > Hi all,
>> >
>> > TL;DR: Should we sign Python artifacts and vote for them too?
>> >
>> > The artifacts built from a commit SHA are to be signed[1] and voted on
>> > for distribution via official release channels[2] downloads.apache.org,
>> > docker[3].
>> >
>> > Some projects choose their policy for specific platforms (pypi.org),
>> > such as Apache Spark publishes pypi package[4] with Infrastructure's
>> > knowledge.
>> >
>> > This said, we have released our build artifacts produced by
>> > maven via `mvn deploy -P'distribution,rat'` for 2.0 release[5] which
>> > does not include our python API.
>> >
>> > Previously SystemML had released[6] the python distribution, along with
>> > other src, bin files.
>> >
>> > Note: Pypi releases, github tags and docker images are convenience
>> packages
>> > and are not needed to go through formal voting.
>> >
>> > [1] https://www.apache.org/legal/release-policy.html#release-signing
>> > [2] https://infra.apache.org/release-distribution.html#unreleased
>> > [3] https://hub.docker.com/u/apache
>> > [4] https://pypi.org/project/pyspark/
>> > [5] https://downloads.apache.org/systemds/2.0.0/
>> > [6] https://downloads.apache.org/systemds/1.2.0/
>> >
>> > Thanks and regards,
>> > Janardhan
>> >
>>
>
Re: [QUESTION] Does the formal voting applies to all the deliverables
from a commit?
Posted by Janardhan <ja...@gmail.com>.
Hi Sebastian,
> Apache have a official docker which you pointed out many times.
> If you feel the need that we use that, feel free to start converting to
it.
> I did not have the time to change it, especially since what we have
currently is working for our needs.
Sorry, it felt like an imposition. This one is not about the docker but
writing down all the release channels that are available to us for record.
The docker feature is tracked at SYSTEMDS-2941.
Thanks,
Janardhan
On Mon, May 24, 2021 at 7:34 PM Matthias Boehm <mb...@gmail.com> wrote:
> thanks for moving this discussion from the PR [1] to the dev list.
>
> Let's start by talking about the current status. First, we do package
> the Python API into the source release (see systemds-2.0.0-src), but not
> the binary release. So related RC votes cover the Python API too. The
> pypi deployment corresponds to the released artifacts, but we allow
> patch updates of the Python API because we currently only release
> major/minor versions but no patches. The need for such patch updates
> originated from different maturity levels of the new Python API and the
> rest of the system.
>
> Personally, I would prefer joint releases where we try to keep the
> deployed Python artifacts consistent with the released main artifacts,
> avoid patch releases of the Python API, so the vote covers both. Similar
> to concerns on the PR, I would not include the Python API into the maven
> build process.
>
> [1] https://github.com/apache/systemds/pull/1225
>
> Regards,
> Matthias
>
> On 5/24/2021 3:33 PM, Janardhan wrote:
> > Hi all,
> >
> > TL;DR: Should we sign Python artifacts and vote for them too?
> >
> > The artifacts built from a commit SHA are to be signed[1] and voted on
> > for distribution via official release channels[2] downloads.apache.org,
> > docker[3].
> >
> > Some projects choose their policy for specific platforms (pypi.org),
> > such as Apache Spark publishes pypi package[4] with Infrastructure's
> > knowledge.
> >
> > This said, we have released our build artifacts produced by
> > maven via `mvn deploy -P'distribution,rat'` for 2.0 release[5] which
> > does not include our python API.
> >
> > Previously SystemML had released[6] the python distribution, along with
> > other src, bin files.
> >
> > Note: Pypi releases, github tags and docker images are convenience
> packages
> > and are not needed to go through formal voting.
> >
> > [1] https://www.apache.org/legal/release-policy.html#release-signing
> > [2] https://infra.apache.org/release-distribution.html#unreleased
> > [3] https://hub.docker.com/u/apache
> > [4] https://pypi.org/project/pyspark/
> > [5] https://downloads.apache.org/systemds/2.0.0/
> > [6] https://downloads.apache.org/systemds/1.2.0/
> >
> > Thanks and regards,
> > Janardhan
> >
>
Re: [QUESTION] Does the formal voting applies to all the deliverables
from a commit?
Posted by Matthias Boehm <mb...@gmail.com>.
thanks for moving this discussion from the PR [1] to the dev list.
Let's start by talking about the current status. First, we do package
the Python API into the source release (see systemds-2.0.0-src), but not
the binary release. So related RC votes cover the Python API too. The
pypi deployment corresponds to the released artifacts, but we allow
patch updates of the Python API because we currently only release
major/minor versions but no patches. The need for such patch updates
originated from different maturity levels of the new Python API and the
rest of the system.
Personally, I would prefer joint releases where we try to keep the
deployed Python artifacts consistent with the released main artifacts,
avoid patch releases of the Python API, so the vote covers both. Similar
to concerns on the PR, I would not include the Python API into the maven
build process.
[1] https://github.com/apache/systemds/pull/1225
Regards,
Matthias
On 5/24/2021 3:33 PM, Janardhan wrote:
> Hi all,
>
> TL;DR: Should we sign Python artifacts and vote for them too?
>
> The artifacts built from a commit SHA are to be signed[1] and voted on
> for distribution via official release channels[2] downloads.apache.org,
> docker[3].
>
> Some projects choose their policy for specific platforms (pypi.org),
> such as Apache Spark publishes pypi package[4] with Infrastructure's
> knowledge.
>
> This said, we have released our build artifacts produced by
> maven via `mvn deploy -P'distribution,rat'` for 2.0 release[5] which
> does not include our python API.
>
> Previously SystemML had released[6] the python distribution, along with
> other src, bin files.
>
> Note: Pypi releases, github tags and docker images are convenience packages
> and are not needed to go through formal voting.
>
> [1] https://www.apache.org/legal/release-policy.html#release-signing
> [2] https://infra.apache.org/release-distribution.html#unreleased
> [3] https://hub.docker.com/u/apache
> [4] https://pypi.org/project/pyspark/
> [5] https://downloads.apache.org/systemds/2.0.0/
> [6] https://downloads.apache.org/systemds/1.2.0/
>
> Thanks and regards,
> Janardhan
>
Re: [QUESTION] Does the formal voting applies to all the deliverables
from a commit?
Posted by "Baunsgaard, Sebastian" <ba...@tugraz.at.INVALID>.
Hi,
No, to include python in the build using maven,
this is important to separate because it is a different project.
Also maven is not designed (at least to my knowledge) to produce python artifacts,
and i therefore don't see a reason why we should diverge from using the python standard libraries
that we used for the last release.
Yes, to be part of the vote of the release,
It does not make much sense to release the python API at other times than our normal distribution.
Therefore when making a release candidate, we also test the python API (we did this last time as well),
but note that the python package can and should work with either source or any release produced.
Apache have a official docker which you pointed out many times.
If you feel the need that we use that, feel free to start converting to it. I did not have the time to change it, especially since what we have currently is working for our needs.
Apache also have official python release yes,
again same argument as above.
Also i don't think we should copy the other projects way of doing things if it leads to significant increased overhead, and even worse loss of control.
Another thing to discuss is the minor fixes we have done and released after official SystemDS 2.0 release.
Since we had some breaking bugs in the python API, that was patched without a general message on our channels.
How do people think we should handle that?
Best regards
Sebastian
________________________________
From: Janardhan <ja...@apache.org>
Sent: Monday, May 24, 2021 3:33:00 PM
To: dev@systemds.apache.org
Subject: [QUESTION] Does the formal voting applies to all the deliverables from a commit?
Hi all,
TL;DR: Should we sign Python artifacts and vote for them too?
The artifacts built from a commit SHA are to be signed[1] and voted on
for distribution via official release channels[2] downloads.apache.org,
docker[3].
Some projects choose their policy for specific platforms (pypi.org),
such as Apache Spark publishes pypi package[4] with Infrastructure's
knowledge.
This said, we have released our build artifacts produced by
maven via `mvn deploy -P'distribution,rat'` for 2.0 release[5] which
does not include our python API.
Previously SystemML had released[6] the python distribution, along with
other src, bin files.
Note: Pypi releases, github tags and docker images are convenience packages
and are not needed to go through formal voting.
[1] https://www.apache.org/legal/release-policy.html#release-signing
[2] https://infra.apache.org/release-distribution.html#unreleased
[3] https://hub.docker.com/u/apache
[4] https://pypi.org/project/pyspark/
[5] https://downloads.apache.org/systemds/2.0.0/
[6] https://downloads.apache.org/systemds/1.2.0/
Thanks and regards,
Janardhan