You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by Janardhan <ja...@apache.org> on 2023/12/04 03:40:25 UTC
Re: [DISCUSS] SystemDS 3.2.0 release
Hi all
A tentative code freeze date is 8th December based on the current jiras and
PR queue.
If any of your PRs or bugs needs attention, please post a note here.
Make sure to include your contributions in the release iterations. If the
functionality or performance is not as much as you expect - if the code
reasonably serves the purpose it is important to merge into the team
codebase.
It is ok, to merge your contributions incrementally in a few commits.
Code review and merge are crucial since the reviewer learns your
implementation and perhaps add design inputs, assist in maintenance.
Best regards
Janardhan
On Thu, Nov 30, 2023, 21:05 Janardhan <ja...@apache.org> wrote:
> Hi all
>
> We are planning for a code freeze at the end of December first week.
>
> Let us some time to document missing functions, & mark the jira resolved
> if they are done.
>
> If you need assistance or help for review please post a note here.
>
> Best regards
> Janardhan
>
> On Mon, Nov 20, 2023, 20:05 Mark Dokter <md...@know-center.at.invalid>
> wrote:
>
>> Hi!
>>
>> Waiting for a major release is fine by me - I'm not dependent on
>> releases in my work ;-)
>>
>> I'll push the changes to a branch on the systemds repo once I have it
>> cleaned up and rebased on main.
>>
>> Cheers,
>> Mark
>>
>> On 17.11.23 04:21, Janardhan wrote:
>> > Hi Mark
>> >
>> > Code freeze: Hoping for the next one/two weeks depending if there are
>> > issues with the features already introduced.
>> >
>> > Can we create a new branch and merge changes? Why not go ahead and
>> create a
>> > new branch in the team repo and merge changes. Let us give some airtime,
>> > for the feature to make sure it is working as expected & no regressions
>> > would be there.
>> >
>> >
>> > We can do tests on that branch, later we can go for a major release.
>> >
>> >
>> > Best Regards
>> > Janardhan
>> >
>> >
>> > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
>> >
>> >> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
>> >> release to include CUDA 12.0 support.
>> >> In that case, either we make this release a major release, or push
>> >> the version upgrade to the next.
>> >>
>> >> Regards,
>> >> Arnab..
>> >>
>> >> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
>> >> .invalid>
>> >> wrote:
>> >>
>> >>> Hi!
>> >>>
>> >>> What's the time frame until release? I have CUDA 12.0 support working
>> >>> but that needs cleanup and testing.
>> >>>
>> >>> Regards, Mark
>> >>>
>> >>> On 16.11.23 09:34, Matthias Boehm wrote:
>> >>>> great - thanks for starting the process Janardhan. @all: we should
>> all
>> >>>> help close some of the old JIRA tasks and fix remaining known issues.
>> >>>>
>> >>>> Regards,
>> >>>> Matthias
>> >>>>
>> >>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>> >>>>> Hi all,
>> >>>>>
>> >>>>> Let us initiate the release preparations. I'd like to mention the
>> >>>>> release date shortly after our discussion in this mail thread.
>> >>>>>
>> >>>>> Let us take this opportunity to wrap up any features recently
>> >>>>> introduced. Hopefully, the features are functional and reasonably
>> >>>>> tested for the release. If not, you can mention here if you would
>> need
>> >>>>> some assistance, and reviews.
>> >>>>>
>> >>>>> You can start with Updating version details in Jira as follows:
>> >>>>>
>> >>>>> 1. Apply version
>> >>>>> Affects Version/s: SystemDS 3.2
>> >>>>> Fix Version/s: SystemDS 3.2
>> >>>>>
>> >>>>> 2. Fill in Components the Jira is related to
>> >>>>> Component/s:
>> >>>>>
>> >>>>> some of component names:
>> >>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>> >>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>> >>>>>
>> >>>>> --
>> >>>>> Helpful links
>> >>>>>
>> >>>>> [1] all Jira components list:
>> >>>>>
>> >>>
>> >>
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>> >>>>> [2] SystemDS Kanban Board:
>> >>>>>
>> >>>
>> >>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>> >>>>> [3] Release Process:
>> >>>>> https://apache.github.io/systemds/site/release-process.html
>> >>>>>
>> >>>>>
>> >>>>> Best Regards,
>> >>>>> Janardhan
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> >>> Senior Researcher
>> >>>
>> >>> Know-Center GmbH
>> >>> Research Center for Data-Driven Business & Big Data Analytics
>> >>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> >>> FN 199 685 f, Commercial register court: LGZ Graz
>> >>>
>> >>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> >>> mdokter@know-center.at
>> >>>
>> >>> > AILEVATOR: Accelerating AI projects to the highest level:
>> >>> www.know-center.at/ailevator
>> >>>
>> >>
>> >
>>
>>
>>
>> --
>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> Senior Researcher
>>
>> Know-Center GmbH
>> Research Center for Data-Driven Business & Big Data Analytics
>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> FN 199 685 f, Commercial register court: LGZ Graz
>>
>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> mdokter@know-center.at
>>
>> > AILEVATOR: Accelerating AI projects to the highest level:
>> www.know-center.at/ailevator
>>
>
Re: [DISCUSS] SystemDS 3.2.0 release
Posted by Badrul Chowdhury <ba...@gmail.com>.
Hi Sebastian,
I can take a stab at upgrading the Hadoop and Spark versions. Here are 2
PRs addressing the same:
1. Hadoop: https://github.com/apache/systemds/pull/1961
2. [WIP] Spark: https://github.com/apache/systemds/pull/1960
I plan to look into [2] over the weekend, but please feel free to take up
the task if you have bandwidth before then.
Thanks,
Badrul
On Mon, 4 Dec 2023 at 03:21, Sebastian Baunsgaard <se...@bgaard.dk> wrote:
> Hi all,
>
> I hate to stop the release process, but we need to update dependencies with
> critical vulnerabilities before we can release anything.
> This include amoung other things Spark and Hadoop.
>
> https://issues.apache.org/jira/browse/SYSTEMDS-3656
> https://issues.apache.org/jira/browse/SYSTEMDS-3655
>
> I do not have time to update these currently, because of other pressing
> issues.
>
> Best regards
> Sebastian
>
> On Mon, 2023-12-04 at 09:10 +0530, Janardhan wrote:
> > Hi all
> >
> > A tentative code freeze date is 8th December based on the current jiras
> and
> > PR queue.
> >
> > If any of your PRs or bugs needs attention, please post a note here.
> >
> > Make sure to include your contributions in the release iterations. If the
> > functionality or performance is not as much as you expect - if the code
> > reasonably serves the purpose it is important to merge into the team
> > codebase.
> >
> > It is ok, to merge your contributions incrementally in a few commits.
> >
> > Code review and merge are crucial since the reviewer learns your
> > implementation and perhaps add design inputs, assist in maintenance.
> >
> >
> > Best regards
> > Janardhan
> >
> >
> > On Thu, Nov 30, 2023, 21:05 Janardhan <ja...@apache.org> wrote:
> >
> > > Hi all
> > >
> > > We are planning for a code freeze at the end of December first week.
> > >
> > > Let us some time to document missing functions, & mark the jira
> resolved
> > > if they are done.
> > >
> > > If you need assistance or help for review please post a note here.
> > >
> > > Best regards
> > > Janardhan
> > >
> > > On Mon, Nov 20, 2023, 20:05 Mark Dokter <mdokter@know-center.at
> .invalid>
> > > wrote:
> > >
> > > > Hi!
> > > >
> > > > Waiting for a major release is fine by me - I'm not dependent on
> > > > releases in my work ;-)
> > > >
> > > > I'll push the changes to a branch on the systemds repo once I have it
> > > > cleaned up and rebased on main.
> > > >
> > > > Cheers,
> > > > Mark
> > > >
> > > > On 17.11.23 04:21, Janardhan wrote:
> > > > > Hi Mark
> > > > >
> > > > > Code freeze: Hoping for the next one/two weeks depending if there
> are
> > > > > issues with the features already introduced.
> > > > >
> > > > > Can we create a new branch and merge changes? Why not go ahead and
> > > > create a
> > > > > new branch in the team repo and merge changes. Let us give some
> airtime,
> > > > > for the feature to make sure it is working as expected & no
> regressions
> > > > > would be there.
> > > > >
> > > > >
> > > > > We can do tests on that branch, later we can go for a major
> release.
> > > > >
> > > > >
> > > > > Best Regards
> > > > > Janardhan
> > > > >
> > > > >
> > > > > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
> wrote:
> > > > >
> > > > > > CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
> major
> > > > > > release to include CUDA 12.0 support.
> > > > > > In that case, either we make this release a major release, or
> push
> > > > > > the version upgrade to the next.
> > > > > >
> > > > > > Regards,
> > > > > > Arnab..
> > > > > >
> > > > > > On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
> mdokter@know-center.at
> > > > > > .invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi!
> > > > > > >
> > > > > > > What's the time frame until release? I have CUDA 12.0 support
> working
> > > > > > > but that needs cleanup and testing.
> > > > > > >
> > > > > > > Regards, Mark
> > > > > > >
> > > > > > > On 16.11.23 09:34, Matthias Boehm wrote:
> > > > > > > > great - thanks for starting the process Janardhan. @all: we
> should
> > > > all
> > > > > > > > help close some of the old JIRA tasks and fix remaining
> known issues.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Matthias
> > > > > > > >
> > > > > > > > On 11/16/2023 3:35 AM, Janardhan wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Let us initiate the release preparations. I'd like to
> mention the
> > > > > > > > > release date shortly after our discussion in this mail
> thread.
> > > > > > > > >
> > > > > > > > > Let us take this opportunity to wrap up any features
> recently
> > > > > > > > > introduced. Hopefully, the features are functional and
> reasonably
> > > > > > > > > tested for the release. If not, you can mention here if
> you would
> > > > need
> > > > > > > > > some assistance, and reviews.
> > > > > > > > >
> > > > > > > > > You can start with Updating version details in Jira as
> follows:
> > > > > > > > >
> > > > > > > > > 1. Apply version
> > > > > > > > > Affects Version/s: SystemDS 3.2
> > > > > > > > > Fix Version/s: SystemDS 3.2
> > > > > > > > >
> > > > > > > > > 2. Fill in Components the Jira is related to
> > > > > > > > > Component/s:
> > > > > > > > >
> > > > > > > > > some of component names:
> > > > > > > > > Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> > > > > > > > > Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Helpful links
> > > > > > > > >
> > > > > > > > > [1] all Jira components list:
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> > > > > > > > > [2] SystemDS Kanban Board:
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> > > > > > > > > [3] Release Process:
> > > > > > > > >
> https://apache.github.io/systemds/site/release-process.html
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Best Regards,
> > > > > > > > > Janardhan
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > > > > > Senior Researcher
> > > > > > >
> > > > > > > Know-Center GmbH
> > > > > > > Research Center for Data-Driven Business & Big Data Analytics
> > > > > > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > > > > > FN 199 685 f, Commercial register court: LGZ Graz
> > > > > > >
> > > > > > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > > > > > mdokter@know-center.at
> > > > > > >
> > > > > > > > AILEVATOR: Accelerating AI projects to the highest level:
> > > > > > > www.know-center.at/ailevator
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > > Senior Researcher
> > > >
> > > > Know-Center GmbH
> > > > Research Center for Data-Driven Business & Big Data Analytics
> > > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > > FN 199 685 f, Commercial register court: LGZ Graz
> > > >
> > > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > > mdokter@know-center.at
> > > >
> > > > > AILEVATOR: Accelerating AI projects to the highest level:
> > > > www.know-center.at/ailevator
> > > >
> > >
>
>
--
Best,
Badrul
Re: [DISCUSS] SystemDS 3.2.0 release
Posted by Sebastian Baunsgaard <se...@bgaard.dk>.
Hi all,
I hate to stop the release process, but we need to update dependencies with
critical vulnerabilities before we can release anything.
This include amoung other things Spark and Hadoop.
https://issues.apache.org/jira/browse/SYSTEMDS-3656
https://issues.apache.org/jira/browse/SYSTEMDS-3655
I do not have time to update these currently, because of other pressing issues.
Best regards
Sebastian
On Mon, 2023-12-04 at 09:10 +0530, Janardhan wrote:
> Hi all
>
> A tentative code freeze date is 8th December based on the current jiras and
> PR queue.
>
> If any of your PRs or bugs needs attention, please post a note here.
>
> Make sure to include your contributions in the release iterations. If the
> functionality or performance is not as much as you expect - if the code
> reasonably serves the purpose it is important to merge into the team
> codebase.
>
> It is ok, to merge your contributions incrementally in a few commits.
>
> Code review and merge are crucial since the reviewer learns your
> implementation and perhaps add design inputs, assist in maintenance.
>
>
> Best regards
> Janardhan
>
>
> On Thu, Nov 30, 2023, 21:05 Janardhan <ja...@apache.org> wrote:
>
> > Hi all
> >
> > We are planning for a code freeze at the end of December first week.
> >
> > Let us some time to document missing functions, & mark the jira resolved
> > if they are done.
> >
> > If you need assistance or help for review please post a note here.
> >
> > Best regards
> > Janardhan
> >
> > On Mon, Nov 20, 2023, 20:05 Mark Dokter <md...@know-center.at.invalid>
> > wrote:
> >
> > > Hi!
> > >
> > > Waiting for a major release is fine by me - I'm not dependent on
> > > releases in my work ;-)
> > >
> > > I'll push the changes to a branch on the systemds repo once I have it
> > > cleaned up and rebased on main.
> > >
> > > Cheers,
> > > Mark
> > >
> > > On 17.11.23 04:21, Janardhan wrote:
> > > > Hi Mark
> > > >
> > > > Code freeze: Hoping for the next one/two weeks depending if there are
> > > > issues with the features already introduced.
> > > >
> > > > Can we create a new branch and merge changes? Why not go ahead and
> > > create a
> > > > new branch in the team repo and merge changes. Let us give some airtime,
> > > > for the feature to make sure it is working as expected & no regressions
> > > > would be there.
> > > >
> > > >
> > > > We can do tests on that branch, later we can go for a major release.
> > > >
> > > >
> > > > Best Regards
> > > > Janardhan
> > > >
> > > >
> > > > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
> > > >
> > > > > CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
> > > > > release to include CUDA 12.0 support.
> > > > > In that case, either we make this release a major release, or push
> > > > > the version upgrade to the next.
> > > > >
> > > > > Regards,
> > > > > Arnab..
> > > > >
> > > > > On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
> > > > > .invalid>
> > > > > wrote:
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > > What's the time frame until release? I have CUDA 12.0 support working
> > > > > > but that needs cleanup and testing.
> > > > > >
> > > > > > Regards, Mark
> > > > > >
> > > > > > On 16.11.23 09:34, Matthias Boehm wrote:
> > > > > > > great - thanks for starting the process Janardhan. @all: we should
> > > all
> > > > > > > help close some of the old JIRA tasks and fix remaining known issues.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Matthias
> > > > > > >
> > > > > > > On 11/16/2023 3:35 AM, Janardhan wrote:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > Let us initiate the release preparations. I'd like to mention the
> > > > > > > > release date shortly after our discussion in this mail thread.
> > > > > > > >
> > > > > > > > Let us take this opportunity to wrap up any features recently
> > > > > > > > introduced. Hopefully, the features are functional and reasonably
> > > > > > > > tested for the release. If not, you can mention here if you would
> > > need
> > > > > > > > some assistance, and reviews.
> > > > > > > >
> > > > > > > > You can start with Updating version details in Jira as follows:
> > > > > > > >
> > > > > > > > 1. Apply version
> > > > > > > > Affects Version/s: SystemDS 3.2
> > > > > > > > Fix Version/s: SystemDS 3.2
> > > > > > > >
> > > > > > > > 2. Fill in Components the Jira is related to
> > > > > > > > Component/s:
> > > > > > > >
> > > > > > > > some of component names:
> > > > > > > > Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> > > > > > > > Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> > > > > > > >
> > > > > > > > --
> > > > > > > > Helpful links
> > > > > > > >
> > > > > > > > [1] all Jira components list:
> > > > > > > >
> > > > > >
> > > > >
> > > https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> > > > > > > > [2] SystemDS Kanban Board:
> > > > > > > >
> > > > > >
> > > > >
> > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> > > > > > > > [3] Release Process:
> > > > > > > > https://apache.github.io/systemds/site/release-process.html
> > > > > > > >
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Janardhan
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > > > > Senior Researcher
> > > > > >
> > > > > > Know-Center GmbH
> > > > > > Research Center for Data-Driven Business & Big Data Analytics
> > > > > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > > > > FN 199 685 f, Commercial register court: LGZ Graz
> > > > > >
> > > > > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > > > > mdokter@know-center.at
> > > > > >
> > > > > > > AILEVATOR: Accelerating AI projects to the highest level:
> > > > > > www.know-center.at/ailevator
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > Senior Researcher
> > >
> > > Know-Center GmbH
> > > Research Center for Data-Driven Business & Big Data Analytics
> > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > FN 199 685 f, Commercial register court: LGZ Graz
> > >
> > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > mdokter@know-center.at
> > >
> > > > AILEVATOR: Accelerating AI projects to the highest level:
> > > www.know-center.at/ailevator
> > >
> >