You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by Janardhan <ja...@apache.org> on 2023/11/16 02:35:43 UTC

[DISCUSS] SystemDS 3.2.0 release

Hi all,

Let us initiate the release preparations. I'd like to mention the
release date shortly after our discussion in this mail thread.

Let us take this opportunity to wrap up any features recently
introduced. Hopefully, the features are functional and reasonably
tested for the release. If not, you can mention here if you would need
some assistance, and reviews.

You can start with Updating version details in Jira as follows:

1. Apply version
    Affects Version/s: SystemDS 3.2
    Fix Version/s: SystemDS 3.2

2. Fill in Components the Jira is related to
    Component/s:

some of component names:
Documentation, Runtime, Compiler, Build, APIs, Algorithms,
Federated, Compression, Builtins, Frame, Matrix, GPU, CLA

--
Helpful links

[1] all Jira components list:
https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
[2] SystemDS Kanban Board:
https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
[3] Release Process:
https://apache.github.io/systemds/site/release-process.html


Best Regards,
Janardhan

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <ja...@apache.org>.
Hi all,

there is a code freeze on the repository now.

Best regards
Janardhan


On Sat, 10 Feb 2024 at 09:20, Janardhan <br...@gmail.com> wrote:

> Hi all, there is a code freeze on the repository now.
>
> Best regards
> Janardhan
>
> On Wed, 7 Feb 2024 at 20:50, Janardhan <ja...@apache.org> wrote:
>
>> Hi team
>>
>> We are planning to create a release candidate this weekend.
>>
>> Please hold any big PRs / code changes that affect internals for now.
>> Let's prioritize bug fixes, adding missing documentation.
>>
>> Please take a moment to close any tickets that are already fixed.
>> Let's keep our github CI tests to green, even for the trivial tests.
>>
>> Unless there are security/license issues, we should not stop the
>> release process.
>>
>> Best regards
>> Janardhan
>>
>> On Mon, 29 Jan 2024 at 01:40, Matthias Boehm <mb...@gmail.com> wrote:
>>
>>> awesome - thanks to both of you. Let's all help to double check things
>>> to avoid unnecessary stops of release candidates.
>>>
>>> Regards,
>>> Matthias
>>>
>>> On 1/28/2024 7:27 PM, arnab phani wrote:
>>> > Thanks, Janardhan, for restarting the process.
>>> > I have the results of the first round of performance tests. They look
>>> fine
>>> > at a glance.
>>> > We will have a detailed look into the results in the next few days.
>>> >
>>> > Regards,
>>> > Arnab..
>>> >
>>> > On Thu, Jan 25, 2024 at 3:11 PM Janardhan <ja...@apache.org>
>>> wrote:
>>> >
>>> >> Hi team
>>> >>
>>> >> The blockers pointed out by Sebastian related the Spark and Hadoop are
>>> >> cleared now.
>>> >>
>>> >> Now, we will cut a release candidate once the performance test
>>> completed.
>>> >>
>>> >> Meanwhile, feel free to go to Jira to see if your features are marked
>>> and
>>> >> completed features are closed with fix version: SystemDS 3.2.0
>>> >>
>>> >> If your code is needs review or help, feel free to ping here or in
>>> the PR
>>> >> my @ mention (do not hesitate).
>>> >>
>>> >> If your feature only needs minor touches, do not wait for the next
>>> release
>>> >> - simply get it into this release version.
>>> >>
>>> >> Best regards
>>> >> Janardhan
>>> >>
>>> >>
>>> >> On Thu, Dec 14, 2023, 20:47 Mark Dokter <mdokter@know-center.at
>>> .invalid>
>>> >> wrote:
>>> >>
>>> >>> Hi!
>>> >>>
>>> >>> I got confused because of the delayed release and the note on Jira
>>> >>> ticket 3375. So everything stays as planned - I'll work it into a
>>> >>> branch. This is the best option anyway as I spotted some unfinished
>>> >>> CUDNN business :-P
>>> >>>
>>> >>> cheers,
>>> >>> Mark
>>> >>>
>>> >>> On 14.12.23 12:42, arnab phani wrote:
>>> >>>> Yes, that sounds like the right thing to do.
>>> >>>> You can prepare the branch already so we can run experiments right
>>> >> after
>>> >>>> this release.
>>> >>>>
>>> >>>> Regards,
>>> >>>> Arnab..
>>> >>>>
>>> >>>> On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <
>>> mdokter@know-center.at
>>> >>> .invalid>
>>> >>>> wrote:
>>> >>>>
>>> >>>>> Hi!
>>> >>>>>
>>> >>>>> To resolve the discussion about the CUDA upgrade: We wait for
>>> SystemDS
>>> >>>>> 4.0 to merge it to the main branch?
>>> >>>>>
>>> >>>>> Regards,
>>> >>>>> Mark
>>> >>>>>
>>> >>>>> On 20.11.23 15:35, Mark Dokter wrote:
>>> >>>>>> Hi!
>>> >>>>>>
>>> >>>>>> Waiting for a major release is fine by me - I'm not dependent on
>>> >>>>>> releases in my work ;-)
>>> >>>>>>
>>> >>>>>> I'll push the changes to a branch on the systemds repo once I
>>> have it
>>> >>>>>> cleaned up and rebased on main.
>>> >>>>>>
>>> >>>>>> Cheers,
>>> >>>>>> Mark
>>> >>>>>>
>>> >>>>>> On 17.11.23 04:21, Janardhan wrote:
>>> >>>>>>> Hi Mark
>>> >>>>>>>
>>> >>>>>>> Code freeze: Hoping for the next one/two weeks depending if there
>>> >> are
>>> >>>>>>> issues with the features already introduced.
>>> >>>>>>>
>>> >>>>>>> Can we create a new branch and merge changes? Why not go ahead
>>> and
>>> >>>>>>> create a
>>> >>>>>>> new branch in the team repo and merge changes. Let us give some
>>> >>> airtime,
>>> >>>>>>> for the feature to make sure it is working as expected & no
>>> >>> regressions
>>> >>>>>>> would be there.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> We can do tests on that branch, later we can go for a major
>>> release.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Best Regards
>>> >>>>>>> Janardhan
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
>>> >> wrote:
>>> >>>>>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
>>> >> major
>>> >>>>>>>> release to include CUDA 12.0 support.
>>> >>>>>>>> In that case, either we make this release a major release, or
>>> push
>>> >>>>>>>> the version upgrade to the next.
>>> >>>>>>>>
>>> >>>>>>>> Regards,
>>> >>>>>>>> Arnab..
>>> >>>>>>>>
>>> >>>>>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
>>> >> mdokter@know-center.at
>>> >>>>>>>> .invalid>
>>> >>>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> Hi!
>>> >>>>>>>>>
>>> >>>>>>>>> What's the time frame until release? I have CUDA 12.0 support
>>> >>> working
>>> >>>>>>>>> but that needs cleanup and testing.
>>> >>>>>>>>>
>>> >>>>>>>>> Regards, Mark
>>> >>>>>>>>>
>>> >>>>>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
>>> >>>>>>>>>> great - thanks for starting the process Janardhan. @all: we
>>> >> should
>>> >>>>> all
>>> >>>>>>>>>> help close some of the old JIRA tasks and fix remaining known
>>> >>> issues.
>>> >>>>>>>>>> Regards,
>>> >>>>>>>>>> Matthias
>>> >>>>>>>>>>
>>> >>>>>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>>> >>>>>>>>>>> Hi all,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Let us initiate the release preparations. I'd like to mention
>>> >> the
>>> >>>>>>>>>>> release date shortly after our discussion in this mail
>>> thread.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Let us take this opportunity to wrap up any features recently
>>> >>>>>>>>>>> introduced. Hopefully, the features are functional and
>>> >> reasonably
>>> >>>>>>>>>>> tested for the release. If not, you can mention here if you
>>> >> would
>>> >>>>>>>>>>> need
>>> >>>>>>>>>>> some assistance, and reviews.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> You can start with Updating version details in Jira as
>>> follows:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> 1. Apply version
>>> >>>>>>>>>>>         Affects Version/s: SystemDS 3.2
>>> >>>>>>>>>>>         Fix Version/s: SystemDS 3.2
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> 2. Fill in Components the Jira is related to
>>> >>>>>>>>>>>         Component/s:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> some of component names:
>>> >>>>>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>>> >>>>>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> --
>>> >>>>>>>>>>> Helpful links
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> [1] all Jira components list:
>>> >>>>>>>>>>>
>>> >>
>>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>>> >>>>>>>>>>> [2] SystemDS Kanban Board:
>>> >>>>>>>>>>>
>>> >>
>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>>> >>>>>>>>>>> [3] Release Process:
>>> >>>>>>>>>>> https://apache.github.io/systemds/site/release-process.html
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Best Regards,
>>> >>>>>>>>>>> Janardhan
>>> >>>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> --
>>> >>>>>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>> >>>>>>>>> Senior Researcher
>>> >>>>>>>>>
>>> >>>>>>>>> Know-Center GmbH
>>> >>>>>>>>> Research Center for Data-Driven Business & Big Data Analytics
>>> >>>>>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>> >>>>>>>>> FN 199 685 f, Commercial register court: LGZ Graz
>>> >>>>>>>>>
>>> >>>>>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>> >>>>>>>>> mdokter@know-center.at
>>> >>>>>>>>>
>>> >>>>>>>>>     > AILEVATOR: Accelerating AI projects to the highest level:
>>> >>>>>>>>> www.know-center.at/ailevator
>>> >>>>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>> >>>>> Senior Researcher
>>> >>>>>
>>> >>>>> Know-Center GmbH
>>> >>>>> Research Center for Data-Driven Business & Big Data Analytics
>>> >>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>> >>>>> FN 199 685 f, Commercial register court: LGZ Graz
>>> >>>>>
>>> >>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>> >>>>> mdokter@know-center.at
>>> >>>>>
>>> >>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
>>> >>>>> www.know-center.at/ailevator
>>> >>>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>> >>> Senior Researcher
>>> >>>
>>> >>> Know-Center GmbH
>>> >>> Research Center for Data-Driven Business & Big Data Analytics
>>> >>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>> >>> FN 199 685 f, Commercial register court: LGZ Graz
>>> >>>
>>> >>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>> >>> mdokter@know-center.at
>>> >>>
>>> >>>   > AILEVATOR: Accelerating AI projects to the highest level:
>>> >>> www.know-center.at/ailevator
>>> >>>
>>>
>>>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <br...@gmail.com>.
Hi all, there is a code freeze on the repository now.

Best regards
Janardhan

On Wed, 7 Feb 2024 at 20:50, Janardhan <ja...@apache.org> wrote:

> Hi team
>
> We are planning to create a release candidate this weekend.
>
> Please hold any big PRs / code changes that affect internals for now.
> Let's prioritize bug fixes, adding missing documentation.
>
> Please take a moment to close any tickets that are already fixed.
> Let's keep our github CI tests to green, even for the trivial tests.
>
> Unless there are security/license issues, we should not stop the
> release process.
>
> Best regards
> Janardhan
>
> On Mon, 29 Jan 2024 at 01:40, Matthias Boehm <mb...@gmail.com> wrote:
>
>> awesome - thanks to both of you. Let's all help to double check things
>> to avoid unnecessary stops of release candidates.
>>
>> Regards,
>> Matthias
>>
>> On 1/28/2024 7:27 PM, arnab phani wrote:
>> > Thanks, Janardhan, for restarting the process.
>> > I have the results of the first round of performance tests. They look
>> fine
>> > at a glance.
>> > We will have a detailed look into the results in the next few days.
>> >
>> > Regards,
>> > Arnab..
>> >
>> > On Thu, Jan 25, 2024 at 3:11 PM Janardhan <ja...@apache.org> wrote:
>> >
>> >> Hi team
>> >>
>> >> The blockers pointed out by Sebastian related the Spark and Hadoop are
>> >> cleared now.
>> >>
>> >> Now, we will cut a release candidate once the performance test
>> completed.
>> >>
>> >> Meanwhile, feel free to go to Jira to see if your features are marked
>> and
>> >> completed features are closed with fix version: SystemDS 3.2.0
>> >>
>> >> If your code is needs review or help, feel free to ping here or in the
>> PR
>> >> my @ mention (do not hesitate).
>> >>
>> >> If your feature only needs minor touches, do not wait for the next
>> release
>> >> - simply get it into this release version.
>> >>
>> >> Best regards
>> >> Janardhan
>> >>
>> >>
>> >> On Thu, Dec 14, 2023, 20:47 Mark Dokter <mdokter@know-center.at
>> .invalid>
>> >> wrote:
>> >>
>> >>> Hi!
>> >>>
>> >>> I got confused because of the delayed release and the note on Jira
>> >>> ticket 3375. So everything stays as planned - I'll work it into a
>> >>> branch. This is the best option anyway as I spotted some unfinished
>> >>> CUDNN business :-P
>> >>>
>> >>> cheers,
>> >>> Mark
>> >>>
>> >>> On 14.12.23 12:42, arnab phani wrote:
>> >>>> Yes, that sounds like the right thing to do.
>> >>>> You can prepare the branch already so we can run experiments right
>> >> after
>> >>>> this release.
>> >>>>
>> >>>> Regards,
>> >>>> Arnab..
>> >>>>
>> >>>> On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <mdokter@know-center.at
>> >>> .invalid>
>> >>>> wrote:
>> >>>>
>> >>>>> Hi!
>> >>>>>
>> >>>>> To resolve the discussion about the CUDA upgrade: We wait for
>> SystemDS
>> >>>>> 4.0 to merge it to the main branch?
>> >>>>>
>> >>>>> Regards,
>> >>>>> Mark
>> >>>>>
>> >>>>> On 20.11.23 15:35, Mark Dokter wrote:
>> >>>>>> Hi!
>> >>>>>>
>> >>>>>> Waiting for a major release is fine by me - I'm not dependent on
>> >>>>>> releases in my work ;-)
>> >>>>>>
>> >>>>>> I'll push the changes to a branch on the systemds repo once I have
>> it
>> >>>>>> cleaned up and rebased on main.
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Mark
>> >>>>>>
>> >>>>>> On 17.11.23 04:21, Janardhan wrote:
>> >>>>>>> Hi Mark
>> >>>>>>>
>> >>>>>>> Code freeze: Hoping for the next one/two weeks depending if there
>> >> are
>> >>>>>>> issues with the features already introduced.
>> >>>>>>>
>> >>>>>>> Can we create a new branch and merge changes? Why not go ahead and
>> >>>>>>> create a
>> >>>>>>> new branch in the team repo and merge changes. Let us give some
>> >>> airtime,
>> >>>>>>> for the feature to make sure it is working as expected & no
>> >>> regressions
>> >>>>>>> would be there.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We can do tests on that branch, later we can go for a major
>> release.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Best Regards
>> >>>>>>> Janardhan
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
>> >> wrote:
>> >>>>>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
>> >> major
>> >>>>>>>> release to include CUDA 12.0 support.
>> >>>>>>>> In that case, either we make this release a major release, or
>> push
>> >>>>>>>> the version upgrade to the next.
>> >>>>>>>>
>> >>>>>>>> Regards,
>> >>>>>>>> Arnab..
>> >>>>>>>>
>> >>>>>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
>> >> mdokter@know-center.at
>> >>>>>>>> .invalid>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi!
>> >>>>>>>>>
>> >>>>>>>>> What's the time frame until release? I have CUDA 12.0 support
>> >>> working
>> >>>>>>>>> but that needs cleanup and testing.
>> >>>>>>>>>
>> >>>>>>>>> Regards, Mark
>> >>>>>>>>>
>> >>>>>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
>> >>>>>>>>>> great - thanks for starting the process Janardhan. @all: we
>> >> should
>> >>>>> all
>> >>>>>>>>>> help close some of the old JIRA tasks and fix remaining known
>> >>> issues.
>> >>>>>>>>>> Regards,
>> >>>>>>>>>> Matthias
>> >>>>>>>>>>
>> >>>>>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>> >>>>>>>>>>> Hi all,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Let us initiate the release preparations. I'd like to mention
>> >> the
>> >>>>>>>>>>> release date shortly after our discussion in this mail thread.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Let us take this opportunity to wrap up any features recently
>> >>>>>>>>>>> introduced. Hopefully, the features are functional and
>> >> reasonably
>> >>>>>>>>>>> tested for the release. If not, you can mention here if you
>> >> would
>> >>>>>>>>>>> need
>> >>>>>>>>>>> some assistance, and reviews.
>> >>>>>>>>>>>
>> >>>>>>>>>>> You can start with Updating version details in Jira as
>> follows:
>> >>>>>>>>>>>
>> >>>>>>>>>>> 1. Apply version
>> >>>>>>>>>>>         Affects Version/s: SystemDS 3.2
>> >>>>>>>>>>>         Fix Version/s: SystemDS 3.2
>> >>>>>>>>>>>
>> >>>>>>>>>>> 2. Fill in Components the Jira is related to
>> >>>>>>>>>>>         Component/s:
>> >>>>>>>>>>>
>> >>>>>>>>>>> some of component names:
>> >>>>>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>> >>>>>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> Helpful links
>> >>>>>>>>>>>
>> >>>>>>>>>>> [1] all Jira components list:
>> >>>>>>>>>>>
>> >>
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>> >>>>>>>>>>> [2] SystemDS Kanban Board:
>> >>>>>>>>>>>
>> >>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>> >>>>>>>>>>> [3] Release Process:
>> >>>>>>>>>>> https://apache.github.io/systemds/site/release-process.html
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> Best Regards,
>> >>>>>>>>>>> Janardhan
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> >>>>>>>>> Senior Researcher
>> >>>>>>>>>
>> >>>>>>>>> Know-Center GmbH
>> >>>>>>>>> Research Center for Data-Driven Business & Big Data Analytics
>> >>>>>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> >>>>>>>>> FN 199 685 f, Commercial register court: LGZ Graz
>> >>>>>>>>>
>> >>>>>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> >>>>>>>>> mdokter@know-center.at
>> >>>>>>>>>
>> >>>>>>>>>     > AILEVATOR: Accelerating AI projects to the highest level:
>> >>>>>>>>> www.know-center.at/ailevator
>> >>>>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> >>>>> Senior Researcher
>> >>>>>
>> >>>>> Know-Center GmbH
>> >>>>> Research Center for Data-Driven Business & Big Data Analytics
>> >>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> >>>>> FN 199 685 f, Commercial register court: LGZ Graz
>> >>>>>
>> >>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> >>>>> mdokter@know-center.at
>> >>>>>
>> >>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
>> >>>>> www.know-center.at/ailevator
>> >>>>>
>> >>>
>> >>>
>> >>> --
>> >>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> >>> Senior Researcher
>> >>>
>> >>> Know-Center GmbH
>> >>> Research Center for Data-Driven Business & Big Data Analytics
>> >>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> >>> FN 199 685 f, Commercial register court: LGZ Graz
>> >>>
>> >>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> >>> mdokter@know-center.at
>> >>>
>> >>>   > AILEVATOR: Accelerating AI projects to the highest level:
>> >>> www.know-center.at/ailevator
>> >>>
>>
>>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <ja...@apache.org>.
Hi team

We are planning to create a release candidate this weekend.

Please hold any big PRs / code changes that affect internals for now.
Let's prioritize bug fixes, adding missing documentation.

Please take a moment to close any tickets that are already fixed.
Let's keep our github CI tests to green, even for the trivial tests.

Unless there are security/license issues, we should not stop the
release process.

Best regards
Janardhan

On Mon, 29 Jan 2024 at 01:40, Matthias Boehm <mb...@gmail.com> wrote:

> awesome - thanks to both of you. Let's all help to double check things
> to avoid unnecessary stops of release candidates.
>
> Regards,
> Matthias
>
> On 1/28/2024 7:27 PM, arnab phani wrote:
> > Thanks, Janardhan, for restarting the process.
> > I have the results of the first round of performance tests. They look
> fine
> > at a glance.
> > We will have a detailed look into the results in the next few days.
> >
> > Regards,
> > Arnab..
> >
> > On Thu, Jan 25, 2024 at 3:11 PM Janardhan <ja...@apache.org> wrote:
> >
> >> Hi team
> >>
> >> The blockers pointed out by Sebastian related the Spark and Hadoop are
> >> cleared now.
> >>
> >> Now, we will cut a release candidate once the performance test
> completed.
> >>
> >> Meanwhile, feel free to go to Jira to see if your features are marked
> and
> >> completed features are closed with fix version: SystemDS 3.2.0
> >>
> >> If your code is needs review or help, feel free to ping here or in the
> PR
> >> my @ mention (do not hesitate).
> >>
> >> If your feature only needs minor touches, do not wait for the next
> release
> >> - simply get it into this release version.
> >>
> >> Best regards
> >> Janardhan
> >>
> >>
> >> On Thu, Dec 14, 2023, 20:47 Mark Dokter <mdokter@know-center.at
> .invalid>
> >> wrote:
> >>
> >>> Hi!
> >>>
> >>> I got confused because of the delayed release and the note on Jira
> >>> ticket 3375. So everything stays as planned - I'll work it into a
> >>> branch. This is the best option anyway as I spotted some unfinished
> >>> CUDNN business :-P
> >>>
> >>> cheers,
> >>> Mark
> >>>
> >>> On 14.12.23 12:42, arnab phani wrote:
> >>>> Yes, that sounds like the right thing to do.
> >>>> You can prepare the branch already so we can run experiments right
> >> after
> >>>> this release.
> >>>>
> >>>> Regards,
> >>>> Arnab..
> >>>>
> >>>> On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <mdokter@know-center.at
> >>> .invalid>
> >>>> wrote:
> >>>>
> >>>>> Hi!
> >>>>>
> >>>>> To resolve the discussion about the CUDA upgrade: We wait for
> SystemDS
> >>>>> 4.0 to merge it to the main branch?
> >>>>>
> >>>>> Regards,
> >>>>> Mark
> >>>>>
> >>>>> On 20.11.23 15:35, Mark Dokter wrote:
> >>>>>> Hi!
> >>>>>>
> >>>>>> Waiting for a major release is fine by me - I'm not dependent on
> >>>>>> releases in my work ;-)
> >>>>>>
> >>>>>> I'll push the changes to a branch on the systemds repo once I have
> it
> >>>>>> cleaned up and rebased on main.
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Mark
> >>>>>>
> >>>>>> On 17.11.23 04:21, Janardhan wrote:
> >>>>>>> Hi Mark
> >>>>>>>
> >>>>>>> Code freeze: Hoping for the next one/two weeks depending if there
> >> are
> >>>>>>> issues with the features already introduced.
> >>>>>>>
> >>>>>>> Can we create a new branch and merge changes? Why not go ahead and
> >>>>>>> create a
> >>>>>>> new branch in the team repo and merge changes. Let us give some
> >>> airtime,
> >>>>>>> for the feature to make sure it is working as expected & no
> >>> regressions
> >>>>>>> would be there.
> >>>>>>>
> >>>>>>>
> >>>>>>> We can do tests on that branch, later we can go for a major
> release.
> >>>>>>>
> >>>>>>>
> >>>>>>> Best Regards
> >>>>>>> Janardhan
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
> >> wrote:
> >>>>>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
> >> major
> >>>>>>>> release to include CUDA 12.0 support.
> >>>>>>>> In that case, either we make this release a major release, or push
> >>>>>>>> the version upgrade to the next.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Arnab..
> >>>>>>>>
> >>>>>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
> >> mdokter@know-center.at
> >>>>>>>> .invalid>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi!
> >>>>>>>>>
> >>>>>>>>> What's the time frame until release? I have CUDA 12.0 support
> >>> working
> >>>>>>>>> but that needs cleanup and testing.
> >>>>>>>>>
> >>>>>>>>> Regards, Mark
> >>>>>>>>>
> >>>>>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
> >>>>>>>>>> great - thanks for starting the process Janardhan. @all: we
> >> should
> >>>>> all
> >>>>>>>>>> help close some of the old JIRA tasks and fix remaining known
> >>> issues.
> >>>>>>>>>> Regards,
> >>>>>>>>>> Matthias
> >>>>>>>>>>
> >>>>>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
> >>>>>>>>>>> Hi all,
> >>>>>>>>>>>
> >>>>>>>>>>> Let us initiate the release preparations. I'd like to mention
> >> the
> >>>>>>>>>>> release date shortly after our discussion in this mail thread.
> >>>>>>>>>>>
> >>>>>>>>>>> Let us take this opportunity to wrap up any features recently
> >>>>>>>>>>> introduced. Hopefully, the features are functional and
> >> reasonably
> >>>>>>>>>>> tested for the release. If not, you can mention here if you
> >> would
> >>>>>>>>>>> need
> >>>>>>>>>>> some assistance, and reviews.
> >>>>>>>>>>>
> >>>>>>>>>>> You can start with Updating version details in Jira as follows:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Apply version
> >>>>>>>>>>>         Affects Version/s: SystemDS 3.2
> >>>>>>>>>>>         Fix Version/s: SystemDS 3.2
> >>>>>>>>>>>
> >>>>>>>>>>> 2. Fill in Components the Jira is related to
> >>>>>>>>>>>         Component/s:
> >>>>>>>>>>>
> >>>>>>>>>>> some of component names:
> >>>>>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> >>>>>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Helpful links
> >>>>>>>>>>>
> >>>>>>>>>>> [1] all Jira components list:
> >>>>>>>>>>>
> >>
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> >>>>>>>>>>> [2] SystemDS Kanban Board:
> >>>>>>>>>>>
> >>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> >>>>>>>>>>> [3] Release Process:
> >>>>>>>>>>> https://apache.github.io/systemds/site/release-process.html
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Janardhan
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >>>>>>>>> Senior Researcher
> >>>>>>>>>
> >>>>>>>>> Know-Center GmbH
> >>>>>>>>> Research Center for Data-Driven Business & Big Data Analytics
> >>>>>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >>>>>>>>> FN 199 685 f, Commercial register court: LGZ Graz
> >>>>>>>>>
> >>>>>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
> >>>>>>>>> mdokter@know-center.at
> >>>>>>>>>
> >>>>>>>>>     > AILEVATOR: Accelerating AI projects to the highest level:
> >>>>>>>>> www.know-center.at/ailevator
> >>>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >>>>> Senior Researcher
> >>>>>
> >>>>> Know-Center GmbH
> >>>>> Research Center for Data-Driven Business & Big Data Analytics
> >>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >>>>> FN 199 685 f, Commercial register court: LGZ Graz
> >>>>>
> >>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
> >>>>> mdokter@know-center.at
> >>>>>
> >>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
> >>>>> www.know-center.at/ailevator
> >>>>>
> >>>
> >>>
> >>> --
> >>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >>> Senior Researcher
> >>>
> >>> Know-Center GmbH
> >>> Research Center for Data-Driven Business & Big Data Analytics
> >>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >>> FN 199 685 f, Commercial register court: LGZ Graz
> >>>
> >>> Phone +43-316-873-30804 Mobile +43-664-88796966
> >>> mdokter@know-center.at
> >>>
> >>>   > AILEVATOR: Accelerating AI projects to the highest level:
> >>> www.know-center.at/ailevator
> >>>
>
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Matthias Boehm <mb...@gmail.com>.
awesome - thanks to both of you. Let's all help to double check things 
to avoid unnecessary stops of release candidates.

Regards,
Matthias

On 1/28/2024 7:27 PM, arnab phani wrote:
> Thanks, Janardhan, for restarting the process.
> I have the results of the first round of performance tests. They look fine
> at a glance.
> We will have a detailed look into the results in the next few days.
>
> Regards,
> Arnab..
>
> On Thu, Jan 25, 2024 at 3:11 PM Janardhan <ja...@apache.org> wrote:
>
>> Hi team
>>
>> The blockers pointed out by Sebastian related the Spark and Hadoop are
>> cleared now.
>>
>> Now, we will cut a release candidate once the performance test completed.
>>
>> Meanwhile, feel free to go to Jira to see if your features are marked and
>> completed features are closed with fix version: SystemDS 3.2.0
>>
>> If your code is needs review or help, feel free to ping here or in the PR
>> my @ mention (do not hesitate).
>>
>> If your feature only needs minor touches, do not wait for the next release
>> - simply get it into this release version.
>>
>> Best regards
>> Janardhan
>>
>>
>> On Thu, Dec 14, 2023, 20:47 Mark Dokter <md...@know-center.at.invalid>
>> wrote:
>>
>>> Hi!
>>>
>>> I got confused because of the delayed release and the note on Jira
>>> ticket 3375. So everything stays as planned - I'll work it into a
>>> branch. This is the best option anyway as I spotted some unfinished
>>> CUDNN business :-P
>>>
>>> cheers,
>>> Mark
>>>
>>> On 14.12.23 12:42, arnab phani wrote:
>>>> Yes, that sounds like the right thing to do.
>>>> You can prepare the branch already so we can run experiments right
>> after
>>>> this release.
>>>>
>>>> Regards,
>>>> Arnab..
>>>>
>>>> On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <mdokter@know-center.at
>>> .invalid>
>>>> wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> To resolve the discussion about the CUDA upgrade: We wait for SystemDS
>>>>> 4.0 to merge it to the main branch?
>>>>>
>>>>> Regards,
>>>>> Mark
>>>>>
>>>>> On 20.11.23 15:35, Mark Dokter wrote:
>>>>>> Hi!
>>>>>>
>>>>>> Waiting for a major release is fine by me - I'm not dependent on
>>>>>> releases in my work ;-)
>>>>>>
>>>>>> I'll push the changes to a branch on the systemds repo once I have it
>>>>>> cleaned up and rebased on main.
>>>>>>
>>>>>> Cheers,
>>>>>> Mark
>>>>>>
>>>>>> On 17.11.23 04:21, Janardhan wrote:
>>>>>>> Hi Mark
>>>>>>>
>>>>>>> Code freeze: Hoping for the next one/two weeks depending if there
>> are
>>>>>>> issues with the features already introduced.
>>>>>>>
>>>>>>> Can we create a new branch and merge changes? Why not go ahead and
>>>>>>> create a
>>>>>>> new branch in the team repo and merge changes. Let us give some
>>> airtime,
>>>>>>> for the feature to make sure it is working as expected & no
>>> regressions
>>>>>>> would be there.
>>>>>>>
>>>>>>>
>>>>>>> We can do tests on that branch, later we can go for a major release.
>>>>>>>
>>>>>>>
>>>>>>> Best Regards
>>>>>>> Janardhan
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
>> wrote:
>>>>>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
>> major
>>>>>>>> release to include CUDA 12.0 support.
>>>>>>>> In that case, either we make this release a major release, or push
>>>>>>>> the version upgrade to the next.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Arnab..
>>>>>>>>
>>>>>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
>> mdokter@know-center.at
>>>>>>>> .invalid>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi!
>>>>>>>>>
>>>>>>>>> What's the time frame until release? I have CUDA 12.0 support
>>> working
>>>>>>>>> but that needs cleanup and testing.
>>>>>>>>>
>>>>>>>>> Regards, Mark
>>>>>>>>>
>>>>>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
>>>>>>>>>> great - thanks for starting the process Janardhan. @all: we
>> should
>>>>> all
>>>>>>>>>> help close some of the old JIRA tasks and fix remaining known
>>> issues.
>>>>>>>>>> Regards,
>>>>>>>>>> Matthias
>>>>>>>>>>
>>>>>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> Let us initiate the release preparations. I'd like to mention
>> the
>>>>>>>>>>> release date shortly after our discussion in this mail thread.
>>>>>>>>>>>
>>>>>>>>>>> Let us take this opportunity to wrap up any features recently
>>>>>>>>>>> introduced. Hopefully, the features are functional and
>> reasonably
>>>>>>>>>>> tested for the release. If not, you can mention here if you
>> would
>>>>>>>>>>> need
>>>>>>>>>>> some assistance, and reviews.
>>>>>>>>>>>
>>>>>>>>>>> You can start with Updating version details in Jira as follows:
>>>>>>>>>>>
>>>>>>>>>>> 1. Apply version
>>>>>>>>>>>         Affects Version/s: SystemDS 3.2
>>>>>>>>>>>         Fix Version/s: SystemDS 3.2
>>>>>>>>>>>
>>>>>>>>>>> 2. Fill in Components the Jira is related to
>>>>>>>>>>>         Component/s:
>>>>>>>>>>>
>>>>>>>>>>> some of component names:
>>>>>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>>>>>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Helpful links
>>>>>>>>>>>
>>>>>>>>>>> [1] all Jira components list:
>>>>>>>>>>>
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>>>>>>>>>>> [2] SystemDS Kanban Board:
>>>>>>>>>>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>>>>>>>>>>> [3] Release Process:
>>>>>>>>>>> https://apache.github.io/systemds/site/release-process.html
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Janardhan
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>>>>>>>> Senior Researcher
>>>>>>>>>
>>>>>>>>> Know-Center GmbH
>>>>>>>>> Research Center for Data-Driven Business & Big Data Analytics
>>>>>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>>>>>>>> FN 199 685 f, Commercial register court: LGZ Graz
>>>>>>>>>
>>>>>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>>>>>>>> mdokter@know-center.at
>>>>>>>>>
>>>>>>>>>     > AILEVATOR: Accelerating AI projects to the highest level:
>>>>>>>>> www.know-center.at/ailevator
>>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>>>> Senior Researcher
>>>>>
>>>>> Know-Center GmbH
>>>>> Research Center for Data-Driven Business & Big Data Analytics
>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>>>> FN 199 685 f, Commercial register court: LGZ Graz
>>>>>
>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>>>> mdokter@know-center.at
>>>>>
>>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
>>>>> www.know-center.at/ailevator
>>>>>
>>>
>>>
>>> --
>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>> Senior Researcher
>>>
>>> Know-Center GmbH
>>> Research Center for Data-Driven Business & Big Data Analytics
>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>> FN 199 685 f, Commercial register court: LGZ Graz
>>>
>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>> mdokter@know-center.at
>>>
>>>   > AILEVATOR: Accelerating AI projects to the highest level:
>>> www.know-center.at/ailevator
>>>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by arnab phani <ph...@gmail.com>.
Thanks, Janardhan, for restarting the process.
I have the results of the first round of performance tests. They look fine
at a glance.
We will have a detailed look into the results in the next few days.

Regards,
Arnab..

On Thu, Jan 25, 2024 at 3:11 PM Janardhan <ja...@apache.org> wrote:

> Hi team
>
> The blockers pointed out by Sebastian related the Spark and Hadoop are
> cleared now.
>
> Now, we will cut a release candidate once the performance test completed.
>
> Meanwhile, feel free to go to Jira to see if your features are marked and
> completed features are closed with fix version: SystemDS 3.2.0
>
> If your code is needs review or help, feel free to ping here or in the PR
> my @ mention (do not hesitate).
>
> If your feature only needs minor touches, do not wait for the next release
> - simply get it into this release version.
>
> Best regards
> Janardhan
>
>
> On Thu, Dec 14, 2023, 20:47 Mark Dokter <md...@know-center.at.invalid>
> wrote:
>
> > Hi!
> >
> > I got confused because of the delayed release and the note on Jira
> > ticket 3375. So everything stays as planned - I'll work it into a
> > branch. This is the best option anyway as I spotted some unfinished
> > CUDNN business :-P
> >
> > cheers,
> > Mark
> >
> > On 14.12.23 12:42, arnab phani wrote:
> > > Yes, that sounds like the right thing to do.
> > > You can prepare the branch already so we can run experiments right
> after
> > > this release.
> > >
> > > Regards,
> > > Arnab..
> > >
> > > On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <mdokter@know-center.at
> > .invalid>
> > > wrote:
> > >
> > >> Hi!
> > >>
> > >> To resolve the discussion about the CUDA upgrade: We wait for SystemDS
> > >> 4.0 to merge it to the main branch?
> > >>
> > >> Regards,
> > >> Mark
> > >>
> > >> On 20.11.23 15:35, Mark Dokter wrote:
> > >>> Hi!
> > >>>
> > >>> Waiting for a major release is fine by me - I'm not dependent on
> > >>> releases in my work ;-)
> > >>>
> > >>> I'll push the changes to a branch on the systemds repo once I have it
> > >>> cleaned up and rebased on main.
> > >>>
> > >>> Cheers,
> > >>> Mark
> > >>>
> > >>> On 17.11.23 04:21, Janardhan wrote:
> > >>>> Hi Mark
> > >>>>
> > >>>> Code freeze: Hoping for the next one/two weeks depending if there
> are
> > >>>> issues with the features already introduced.
> > >>>>
> > >>>> Can we create a new branch and merge changes? Why not go ahead and
> > >>>> create a
> > >>>> new branch in the team repo and merge changes. Let us give some
> > airtime,
> > >>>> for the feature to make sure it is working as expected & no
> > regressions
> > >>>> would be there.
> > >>>>
> > >>>>
> > >>>> We can do tests on that branch, later we can go for a major release.
> > >>>>
> > >>>>
> > >>>> Best Regards
> > >>>> Janardhan
> > >>>>
> > >>>>
> > >>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
> wrote:
> > >>>>
> > >>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
> major
> > >>>>> release to include CUDA 12.0 support.
> > >>>>> In that case, either we make this release a major release, or push
> > >>>>> the version upgrade to the next.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Arnab..
> > >>>>>
> > >>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
> mdokter@know-center.at
> > >>>>> .invalid>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi!
> > >>>>>>
> > >>>>>> What's the time frame until release? I have CUDA 12.0 support
> > working
> > >>>>>> but that needs cleanup and testing.
> > >>>>>>
> > >>>>>> Regards, Mark
> > >>>>>>
> > >>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
> > >>>>>>> great - thanks for starting the process Janardhan. @all: we
> should
> > >> all
> > >>>>>>> help close some of the old JIRA tasks and fix remaining known
> > issues.
> > >>>>>>>
> > >>>>>>> Regards,
> > >>>>>>> Matthias
> > >>>>>>>
> > >>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
> > >>>>>>>> Hi all,
> > >>>>>>>>
> > >>>>>>>> Let us initiate the release preparations. I'd like to mention
> the
> > >>>>>>>> release date shortly after our discussion in this mail thread.
> > >>>>>>>>
> > >>>>>>>> Let us take this opportunity to wrap up any features recently
> > >>>>>>>> introduced. Hopefully, the features are functional and
> reasonably
> > >>>>>>>> tested for the release. If not, you can mention here if you
> would
> > >>>>>>>> need
> > >>>>>>>> some assistance, and reviews.
> > >>>>>>>>
> > >>>>>>>> You can start with Updating version details in Jira as follows:
> > >>>>>>>>
> > >>>>>>>> 1. Apply version
> > >>>>>>>>        Affects Version/s: SystemDS 3.2
> > >>>>>>>>        Fix Version/s: SystemDS 3.2
> > >>>>>>>>
> > >>>>>>>> 2. Fill in Components the Jira is related to
> > >>>>>>>>        Component/s:
> > >>>>>>>>
> > >>>>>>>> some of component names:
> > >>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> > >>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Helpful links
> > >>>>>>>>
> > >>>>>>>> [1] all Jira components list:
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>
> >
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> > >>>>>>>> [2] SystemDS Kanban Board:
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> > >>>>>>>> [3] Release Process:
> > >>>>>>>> https://apache.github.io/systemds/site/release-process.html
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Best Regards,
> > >>>>>>>> Janardhan
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > >>>>>> Senior Researcher
> > >>>>>>
> > >>>>>> Know-Center GmbH
> > >>>>>> Research Center for Data-Driven Business & Big Data Analytics
> > >>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> > >>>>>> FN 199 685 f, Commercial register court: LGZ Graz
> > >>>>>>
> > >>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
> > >>>>>> mdokter@know-center.at
> > >>>>>>
> > >>>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
> > >>>>>> www.know-center.at/ailevator
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > >> Senior Researcher
> > >>
> > >> Know-Center GmbH
> > >> Research Center for Data-Driven Business & Big Data Analytics
> > >> Sandgasse 36, 4th floor, 8010 Graz, Austria
> > >> FN 199 685 f, Commercial register court: LGZ Graz
> > >>
> > >> Phone +43-316-873-30804 Mobile +43-664-88796966
> > >> mdokter@know-center.at
> > >>
> > >>   > AILEVATOR: Accelerating AI projects to the highest level:
> > >> www.know-center.at/ailevator
> > >>
> > >
> >
> >
> >
> > --
> > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > Senior Researcher
> >
> > Know-Center GmbH
> > Research Center for Data-Driven Business & Big Data Analytics
> > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > FN 199 685 f, Commercial register court: LGZ Graz
> >
> > Phone +43-316-873-30804 Mobile +43-664-88796966
> > mdokter@know-center.at
> >
> >  > AILEVATOR: Accelerating AI projects to the highest level:
> > www.know-center.at/ailevator
> >
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <ja...@apache.org>.
Hi team

The blockers pointed out by Sebastian related the Spark and Hadoop are
cleared now.

Now, we will cut a release candidate once the performance test completed.

Meanwhile, feel free to go to Jira to see if your features are marked and
completed features are closed with fix version: SystemDS 3.2.0

If your code is needs review or help, feel free to ping here or in the PR
my @ mention (do not hesitate).

If your feature only needs minor touches, do not wait for the next release
- simply get it into this release version.

Best regards
Janardhan


On Thu, Dec 14, 2023, 20:47 Mark Dokter <md...@know-center.at.invalid>
wrote:

> Hi!
>
> I got confused because of the delayed release and the note on Jira
> ticket 3375. So everything stays as planned - I'll work it into a
> branch. This is the best option anyway as I spotted some unfinished
> CUDNN business :-P
>
> cheers,
> Mark
>
> On 14.12.23 12:42, arnab phani wrote:
> > Yes, that sounds like the right thing to do.
> > You can prepare the branch already so we can run experiments right after
> > this release.
> >
> > Regards,
> > Arnab..
> >
> > On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <mdokter@know-center.at
> .invalid>
> > wrote:
> >
> >> Hi!
> >>
> >> To resolve the discussion about the CUDA upgrade: We wait for SystemDS
> >> 4.0 to merge it to the main branch?
> >>
> >> Regards,
> >> Mark
> >>
> >> On 20.11.23 15:35, Mark Dokter wrote:
> >>> Hi!
> >>>
> >>> Waiting for a major release is fine by me - I'm not dependent on
> >>> releases in my work ;-)
> >>>
> >>> I'll push the changes to a branch on the systemds repo once I have it
> >>> cleaned up and rebased on main.
> >>>
> >>> Cheers,
> >>> Mark
> >>>
> >>> On 17.11.23 04:21, Janardhan wrote:
> >>>> Hi Mark
> >>>>
> >>>> Code freeze: Hoping for the next one/two weeks depending if there are
> >>>> issues with the features already introduced.
> >>>>
> >>>> Can we create a new branch and merge changes? Why not go ahead and
> >>>> create a
> >>>> new branch in the team repo and merge changes. Let us give some
> airtime,
> >>>> for the feature to make sure it is working as expected & no
> regressions
> >>>> would be there.
> >>>>
> >>>>
> >>>> We can do tests on that branch, later we can go for a major release.
> >>>>
> >>>>
> >>>> Best Regards
> >>>> Janardhan
> >>>>
> >>>>
> >>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
> >>>>
> >>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
> >>>>> release to include CUDA 12.0 support.
> >>>>> In that case, either we make this release a major release, or push
> >>>>> the version upgrade to the next.
> >>>>>
> >>>>> Regards,
> >>>>> Arnab..
> >>>>>
> >>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
> >>>>> .invalid>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi!
> >>>>>>
> >>>>>> What's the time frame until release? I have CUDA 12.0 support
> working
> >>>>>> but that needs cleanup and testing.
> >>>>>>
> >>>>>> Regards, Mark
> >>>>>>
> >>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
> >>>>>>> great - thanks for starting the process Janardhan. @all: we should
> >> all
> >>>>>>> help close some of the old JIRA tasks and fix remaining known
> issues.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Matthias
> >>>>>>>
> >>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> Let us initiate the release preparations. I'd like to mention the
> >>>>>>>> release date shortly after our discussion in this mail thread.
> >>>>>>>>
> >>>>>>>> Let us take this opportunity to wrap up any features recently
> >>>>>>>> introduced. Hopefully, the features are functional and reasonably
> >>>>>>>> tested for the release. If not, you can mention here if you would
> >>>>>>>> need
> >>>>>>>> some assistance, and reviews.
> >>>>>>>>
> >>>>>>>> You can start with Updating version details in Jira as follows:
> >>>>>>>>
> >>>>>>>> 1. Apply version
> >>>>>>>>        Affects Version/s: SystemDS 3.2
> >>>>>>>>        Fix Version/s: SystemDS 3.2
> >>>>>>>>
> >>>>>>>> 2. Fill in Components the Jira is related to
> >>>>>>>>        Component/s:
> >>>>>>>>
> >>>>>>>> some of component names:
> >>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> >>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Helpful links
> >>>>>>>>
> >>>>>>>> [1] all Jira components list:
> >>>>>>>>
> >>>>>>
> >>>>>
> >>
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> >>>>>>>> [2] SystemDS Kanban Board:
> >>>>>>>>
> >>>>>>
> >>>>>
> >>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> >>>>>>>> [3] Release Process:
> >>>>>>>> https://apache.github.io/systemds/site/release-process.html
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>> Janardhan
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >>>>>> Senior Researcher
> >>>>>>
> >>>>>> Know-Center GmbH
> >>>>>> Research Center for Data-Driven Business & Big Data Analytics
> >>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >>>>>> FN 199 685 f, Commercial register court: LGZ Graz
> >>>>>>
> >>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
> >>>>>> mdokter@know-center.at
> >>>>>>
> >>>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
> >>>>>> www.know-center.at/ailevator
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >> Senior Researcher
> >>
> >> Know-Center GmbH
> >> Research Center for Data-Driven Business & Big Data Analytics
> >> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >> FN 199 685 f, Commercial register court: LGZ Graz
> >>
> >> Phone +43-316-873-30804 Mobile +43-664-88796966
> >> mdokter@know-center.at
> >>
> >>   > AILEVATOR: Accelerating AI projects to the highest level:
> >> www.know-center.at/ailevator
> >>
> >
>
>
>
> --
> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> Senior Researcher
>
> Know-Center GmbH
> Research Center for Data-Driven Business & Big Data Analytics
> Sandgasse 36, 4th floor, 8010 Graz, Austria
> FN 199 685 f, Commercial register court: LGZ Graz
>
> Phone +43-316-873-30804 Mobile +43-664-88796966
> mdokter@know-center.at
>
>  > AILEVATOR: Accelerating AI projects to the highest level:
> www.know-center.at/ailevator
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Mark Dokter <md...@know-center.at.INVALID>.
Hi!

I got confused because of the delayed release and the note on Jira 
ticket 3375. So everything stays as planned - I'll work it into a 
branch. This is the best option anyway as I spotted some unfinished 
CUDNN business :-P

cheers,
Mark

On 14.12.23 12:42, arnab phani wrote:
> Yes, that sounds like the right thing to do.
> You can prepare the branch already so we can run experiments right after
> this release.
> 
> Regards,
> Arnab..
> 
> On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <md...@know-center.at.invalid>
> wrote:
> 
>> Hi!
>>
>> To resolve the discussion about the CUDA upgrade: We wait for SystemDS
>> 4.0 to merge it to the main branch?
>>
>> Regards,
>> Mark
>>
>> On 20.11.23 15:35, Mark Dokter wrote:
>>> Hi!
>>>
>>> Waiting for a major release is fine by me - I'm not dependent on
>>> releases in my work ;-)
>>>
>>> I'll push the changes to a branch on the systemds repo once I have it
>>> cleaned up and rebased on main.
>>>
>>> Cheers,
>>> Mark
>>>
>>> On 17.11.23 04:21, Janardhan wrote:
>>>> Hi Mark
>>>>
>>>> Code freeze: Hoping for the next one/two weeks depending if there are
>>>> issues with the features already introduced.
>>>>
>>>> Can we create a new branch and merge changes? Why not go ahead and
>>>> create a
>>>> new branch in the team repo and merge changes. Let us give some airtime,
>>>> for the feature to make sure it is working as expected & no regressions
>>>> would be there.
>>>>
>>>>
>>>> We can do tests on that branch, later we can go for a major release.
>>>>
>>>>
>>>> Best Regards
>>>> Janardhan
>>>>
>>>>
>>>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
>>>>
>>>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
>>>>> release to include CUDA 12.0 support.
>>>>> In that case, either we make this release a major release, or push
>>>>> the version upgrade to the next.
>>>>>
>>>>> Regards,
>>>>> Arnab..
>>>>>
>>>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
>>>>> .invalid>
>>>>> wrote:
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> What's the time frame until release? I have CUDA 12.0 support working
>>>>>> but that needs cleanup and testing.
>>>>>>
>>>>>> Regards, Mark
>>>>>>
>>>>>> On 16.11.23 09:34, Matthias Boehm wrote:
>>>>>>> great - thanks for starting the process Janardhan. @all: we should
>> all
>>>>>>> help close some of the old JIRA tasks and fix remaining known issues.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Matthias
>>>>>>>
>>>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> Let us initiate the release preparations. I'd like to mention the
>>>>>>>> release date shortly after our discussion in this mail thread.
>>>>>>>>
>>>>>>>> Let us take this opportunity to wrap up any features recently
>>>>>>>> introduced. Hopefully, the features are functional and reasonably
>>>>>>>> tested for the release. If not, you can mention here if you would
>>>>>>>> need
>>>>>>>> some assistance, and reviews.
>>>>>>>>
>>>>>>>> You can start with Updating version details in Jira as follows:
>>>>>>>>
>>>>>>>> 1. Apply version
>>>>>>>>        Affects Version/s: SystemDS 3.2
>>>>>>>>        Fix Version/s: SystemDS 3.2
>>>>>>>>
>>>>>>>> 2. Fill in Components the Jira is related to
>>>>>>>>        Component/s:
>>>>>>>>
>>>>>>>> some of component names:
>>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>>>>>>>>
>>>>>>>> --
>>>>>>>> Helpful links
>>>>>>>>
>>>>>>>> [1] all Jira components list:
>>>>>>>>
>>>>>>
>>>>>
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>>>>>>>> [2] SystemDS Kanban Board:
>>>>>>>>
>>>>>>
>>>>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>>>>>>>> [3] Release Process:
>>>>>>>> https://apache.github.io/systemds/site/release-process.html
>>>>>>>>
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Janardhan
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>>>>> Senior Researcher
>>>>>>
>>>>>> Know-Center GmbH
>>>>>> Research Center for Data-Driven Business & Big Data Analytics
>>>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>>>>> FN 199 685 f, Commercial register court: LGZ Graz
>>>>>>
>>>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>>>>> mdokter@know-center.at
>>>>>>
>>>>>>    > AILEVATOR: Accelerating AI projects to the highest level:
>>>>>> www.know-center.at/ailevator
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> Senior Researcher
>>
>> Know-Center GmbH
>> Research Center for Data-Driven Business & Big Data Analytics
>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> FN 199 685 f, Commercial register court: LGZ Graz
>>
>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> mdokter@know-center.at
>>
>>   > AILEVATOR: Accelerating AI projects to the highest level:
>> www.know-center.at/ailevator
>>
> 



-- 
Dipl.-Ing. Dr.techn. Mark Dokter, BSc
Senior Researcher

Know-Center GmbH
Research Center for Data-Driven Business & Big Data Analytics
Sandgasse 36, 4th floor, 8010 Graz, Austria
FN 199 685 f, Commercial register court: LGZ Graz

Phone +43-316-873-30804 Mobile +43-664-88796966
mdokter@know-center.at

 > AILEVATOR: Accelerating AI projects to the highest level: 
www.know-center.at/ailevator

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by arnab phani <ph...@gmail.com>.
Yes, that sounds like the right thing to do.
You can prepare the branch already so we can run experiments right after
this release.

Regards,
Arnab..

On Thu, Dec 14, 2023 at 12:11 PM Mark Dokter <md...@know-center.at.invalid>
wrote:

> Hi!
>
> To resolve the discussion about the CUDA upgrade: We wait for SystemDS
> 4.0 to merge it to the main branch?
>
> Regards,
> Mark
>
> On 20.11.23 15:35, Mark Dokter wrote:
> > Hi!
> >
> > Waiting for a major release is fine by me - I'm not dependent on
> > releases in my work ;-)
> >
> > I'll push the changes to a branch on the systemds repo once I have it
> > cleaned up and rebased on main.
> >
> > Cheers,
> > Mark
> >
> > On 17.11.23 04:21, Janardhan wrote:
> >> Hi Mark
> >>
> >> Code freeze: Hoping for the next one/two weeks depending if there are
> >> issues with the features already introduced.
> >>
> >> Can we create a new branch and merge changes? Why not go ahead and
> >> create a
> >> new branch in the team repo and merge changes. Let us give some airtime,
> >> for the feature to make sure it is working as expected & no regressions
> >> would be there.
> >>
> >>
> >> We can do tests on that branch, later we can go for a major release.
> >>
> >>
> >> Best Regards
> >> Janardhan
> >>
> >>
> >> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
> >>
> >>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
> >>> release to include CUDA 12.0 support.
> >>> In that case, either we make this release a major release, or push
> >>> the version upgrade to the next.
> >>>
> >>> Regards,
> >>> Arnab..
> >>>
> >>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
> >>> .invalid>
> >>> wrote:
> >>>
> >>>> Hi!
> >>>>
> >>>> What's the time frame until release? I have CUDA 12.0 support working
> >>>> but that needs cleanup and testing.
> >>>>
> >>>> Regards, Mark
> >>>>
> >>>> On 16.11.23 09:34, Matthias Boehm wrote:
> >>>>> great - thanks for starting the process Janardhan. @all: we should
> all
> >>>>> help close some of the old JIRA tasks and fix remaining known issues.
> >>>>>
> >>>>> Regards,
> >>>>> Matthias
> >>>>>
> >>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> Let us initiate the release preparations. I'd like to mention the
> >>>>>> release date shortly after our discussion in this mail thread.
> >>>>>>
> >>>>>> Let us take this opportunity to wrap up any features recently
> >>>>>> introduced. Hopefully, the features are functional and reasonably
> >>>>>> tested for the release. If not, you can mention here if you would
> >>>>>> need
> >>>>>> some assistance, and reviews.
> >>>>>>
> >>>>>> You can start with Updating version details in Jira as follows:
> >>>>>>
> >>>>>> 1. Apply version
> >>>>>>       Affects Version/s: SystemDS 3.2
> >>>>>>       Fix Version/s: SystemDS 3.2
> >>>>>>
> >>>>>> 2. Fill in Components the Jira is related to
> >>>>>>       Component/s:
> >>>>>>
> >>>>>> some of component names:
> >>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> >>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> >>>>>>
> >>>>>> --
> >>>>>> Helpful links
> >>>>>>
> >>>>>> [1] all Jira components list:
> >>>>>>
> >>>>
> >>>
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> >>>>>> [2] SystemDS Kanban Board:
> >>>>>>
> >>>>
> >>>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> >>>>>> [3] Release Process:
> >>>>>> https://apache.github.io/systemds/site/release-process.html
> >>>>>>
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Janardhan
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >>>> Senior Researcher
> >>>>
> >>>> Know-Center GmbH
> >>>> Research Center for Data-Driven Business & Big Data Analytics
> >>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >>>> FN 199 685 f, Commercial register court: LGZ Graz
> >>>>
> >>>> Phone +43-316-873-30804 Mobile +43-664-88796966
> >>>> mdokter@know-center.at
> >>>>
> >>>>   > AILEVATOR: Accelerating AI projects to the highest level:
> >>>> www.know-center.at/ailevator
> >>>>
> >>>
> >>
> >
> >
> >
>
>
>
> --
> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> Senior Researcher
>
> Know-Center GmbH
> Research Center for Data-Driven Business & Big Data Analytics
> Sandgasse 36, 4th floor, 8010 Graz, Austria
> FN 199 685 f, Commercial register court: LGZ Graz
>
> Phone +43-316-873-30804 Mobile +43-664-88796966
> mdokter@know-center.at
>
>  > AILEVATOR: Accelerating AI projects to the highest level:
> www.know-center.at/ailevator
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Mark Dokter <md...@know-center.at.INVALID>.
Hi!

To resolve the discussion about the CUDA upgrade: We wait for SystemDS 
4.0 to merge it to the main branch?

Regards,
Mark

On 20.11.23 15:35, Mark Dokter wrote:
> Hi!
> 
> Waiting for a major release is fine by me - I'm not dependent on 
> releases in my work ;-)
> 
> I'll push the changes to a branch on the systemds repo once I have it 
> cleaned up and rebased on main.
> 
> Cheers,
> Mark
> 
> On 17.11.23 04:21, Janardhan wrote:
>> Hi Mark
>>
>> Code freeze: Hoping for the next one/two weeks depending if there are
>> issues with the features already introduced.
>>
>> Can we create a new branch and merge changes? Why not go ahead and 
>> create a
>> new branch in the team repo and merge changes. Let us give some airtime,
>> for the feature to make sure it is working as expected & no regressions
>> would be there.
>>
>>
>> We can do tests on that branch, later we can go for a major release.
>>
>>
>> Best Regards
>> Janardhan
>>
>>
>> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
>>
>>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
>>> release to include CUDA 12.0 support.
>>> In that case, either we make this release a major release, or push
>>> the version upgrade to the next.
>>>
>>> Regards,
>>> Arnab..
>>>
>>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
>>> .invalid>
>>> wrote:
>>>
>>>> Hi!
>>>>
>>>> What's the time frame until release? I have CUDA 12.0 support working
>>>> but that needs cleanup and testing.
>>>>
>>>> Regards, Mark
>>>>
>>>> On 16.11.23 09:34, Matthias Boehm wrote:
>>>>> great - thanks for starting the process Janardhan. @all: we should all
>>>>> help close some of the old JIRA tasks and fix remaining known issues.
>>>>>
>>>>> Regards,
>>>>> Matthias
>>>>>
>>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> Let us initiate the release preparations. I'd like to mention the
>>>>>> release date shortly after our discussion in this mail thread.
>>>>>>
>>>>>> Let us take this opportunity to wrap up any features recently
>>>>>> introduced. Hopefully, the features are functional and reasonably
>>>>>> tested for the release. If not, you can mention here if you would 
>>>>>> need
>>>>>> some assistance, and reviews.
>>>>>>
>>>>>> You can start with Updating version details in Jira as follows:
>>>>>>
>>>>>> 1. Apply version
>>>>>>       Affects Version/s: SystemDS 3.2
>>>>>>       Fix Version/s: SystemDS 3.2
>>>>>>
>>>>>> 2. Fill in Components the Jira is related to
>>>>>>       Component/s:
>>>>>>
>>>>>> some of component names:
>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>>>>>>
>>>>>> -- 
>>>>>> Helpful links
>>>>>>
>>>>>> [1] all Jira components list:
>>>>>>
>>>>
>>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>>>>>> [2] SystemDS Kanban Board:
>>>>>>
>>>>
>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>>>>>> [3] Release Process:
>>>>>> https://apache.github.io/systemds/site/release-process.html
>>>>>>
>>>>>>
>>>>>> Best Regards,
>>>>>> Janardhan
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>>> Senior Researcher
>>>>
>>>> Know-Center GmbH
>>>> Research Center for Data-Driven Business & Big Data Analytics
>>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>>> FN 199 685 f, Commercial register court: LGZ Graz
>>>>
>>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>>> mdokter@know-center.at
>>>>
>>>>   > AILEVATOR: Accelerating AI projects to the highest level:
>>>> www.know-center.at/ailevator
>>>>
>>>
>>
> 
> 
> 



-- 
Dipl.-Ing. Dr.techn. Mark Dokter, BSc
Senior Researcher

Know-Center GmbH
Research Center for Data-Driven Business & Big Data Analytics
Sandgasse 36, 4th floor, 8010 Graz, Austria
FN 199 685 f, Commercial register court: LGZ Graz

Phone +43-316-873-30804 Mobile +43-664-88796966
mdokter@know-center.at

 > AILEVATOR: Accelerating AI projects to the highest level: 
www.know-center.at/ailevator

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Badrul Chowdhury <ba...@gmail.com>.
Hi Sebastian,

I can take a stab at upgrading the Hadoop and Spark versions. Here are 2
PRs addressing the same:
1. Hadoop: https://github.com/apache/systemds/pull/1961
2. [WIP] Spark: https://github.com/apache/systemds/pull/1960

I plan to look into [2] over the weekend, but please feel free to take up
the task if you have bandwidth before then.


Thanks,
Badrul

On Mon, 4 Dec 2023 at 03:21, Sebastian Baunsgaard <se...@bgaard.dk> wrote:

> Hi all,
>
> I hate to stop the release process, but we need to update dependencies with
> critical vulnerabilities before we can release anything.
> This include amoung other things Spark and Hadoop.
>
> https://issues.apache.org/jira/browse/SYSTEMDS-3656
> https://issues.apache.org/jira/browse/SYSTEMDS-3655
>
> I do not have time to update these currently, because of other pressing
> issues.
>
> Best regards
> Sebastian
>
> On Mon, 2023-12-04 at 09:10 +0530, Janardhan wrote:
> > Hi all
> >
> > A tentative code freeze date is 8th December based on the current jiras
> and
> > PR queue.
> >
> > If any of your PRs or bugs needs attention, please post a note here.
> >
> > Make sure to include your contributions in the release iterations. If the
> > functionality or performance is not as much as you expect - if the code
> > reasonably serves the purpose it is important to merge into the team
> > codebase.
> >
> > It is ok, to merge your contributions incrementally in a few commits.
> >
> > Code review and merge are crucial since the reviewer learns your
> > implementation and perhaps add design inputs, assist in maintenance.
> >
> >
> > Best regards
> > Janardhan
> >
> >
> > On Thu, Nov 30, 2023, 21:05 Janardhan <ja...@apache.org> wrote:
> >
> > > Hi all
> > >
> > > We are planning for a code freeze at the end of December first week.
> > >
> > > Let us some time to document missing functions, & mark the jira
> resolved
> > > if they are done.
> > >
> > > If you need assistance or help for review please post a note here.
> > >
> > > Best regards
> > > Janardhan
> > >
> > > On Mon, Nov 20, 2023, 20:05 Mark Dokter <mdokter@know-center.at
> .invalid>
> > > wrote:
> > >
> > > > Hi!
> > > >
> > > > Waiting for a major release is fine by me - I'm not dependent on
> > > > releases in my work ;-)
> > > >
> > > > I'll push the changes to a branch on the systemds repo once I have it
> > > > cleaned up and rebased on main.
> > > >
> > > > Cheers,
> > > > Mark
> > > >
> > > > On 17.11.23 04:21, Janardhan wrote:
> > > > > Hi Mark
> > > > >
> > > > > Code freeze: Hoping for the next one/two weeks depending if there
> are
> > > > > issues with the features already introduced.
> > > > >
> > > > > Can we create a new branch and merge changes? Why not go ahead and
> > > > create a
> > > > > new branch in the team repo and merge changes. Let us give some
> airtime,
> > > > > for the feature to make sure it is working as expected & no
> regressions
> > > > > would be there.
> > > > >
> > > > >
> > > > > We can do tests on that branch, later we can go for a major
> release.
> > > > >
> > > > >
> > > > > Best Regards
> > > > > Janardhan
> > > > >
> > > > >
> > > > > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com>
> wrote:
> > > > >
> > > > > > CUDA 12.0 is not backward compatible with 10.0, i.e. we need a
> major
> > > > > > release to include CUDA 12.0 support.
> > > > > > In that case, either we make this release a major release, or
> push
> > > > > > the version upgrade to the next.
> > > > > >
> > > > > > Regards,
> > > > > > Arnab..
> > > > > >
> > > > > > On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <
> mdokter@know-center.at
> > > > > > .invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi!
> > > > > > >
> > > > > > > What's the time frame until release? I have CUDA 12.0 support
> working
> > > > > > > but that needs cleanup and testing.
> > > > > > >
> > > > > > > Regards, Mark
> > > > > > >
> > > > > > > On 16.11.23 09:34, Matthias Boehm wrote:
> > > > > > > > great - thanks for starting the process Janardhan. @all: we
> should
> > > > all
> > > > > > > > help close some of the old JIRA tasks and fix remaining
> known issues.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Matthias
> > > > > > > >
> > > > > > > > On 11/16/2023 3:35 AM, Janardhan wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Let us initiate the release preparations. I'd like to
> mention the
> > > > > > > > > release date shortly after our discussion in this mail
> thread.
> > > > > > > > >
> > > > > > > > > Let us take this opportunity to wrap up any features
> recently
> > > > > > > > > introduced. Hopefully, the features are functional and
> reasonably
> > > > > > > > > tested for the release. If not, you can mention here if
> you would
> > > > need
> > > > > > > > > some assistance, and reviews.
> > > > > > > > >
> > > > > > > > > You can start with Updating version details in Jira as
> follows:
> > > > > > > > >
> > > > > > > > > 1. Apply version
> > > > > > > > >       Affects Version/s: SystemDS 3.2
> > > > > > > > >       Fix Version/s: SystemDS 3.2
> > > > > > > > >
> > > > > > > > > 2. Fill in Components the Jira is related to
> > > > > > > > >       Component/s:
> > > > > > > > >
> > > > > > > > > some of component names:
> > > > > > > > > Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> > > > > > > > > Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Helpful links
> > > > > > > > >
> > > > > > > > > [1] all Jira components list:
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> > > > > > > > > [2] SystemDS Kanban Board:
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> > > > > > > > > [3] Release Process:
> > > > > > > > >
> https://apache.github.io/systemds/site/release-process.html
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Best Regards,
> > > > > > > > > Janardhan
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > > > > > Senior Researcher
> > > > > > >
> > > > > > > Know-Center GmbH
> > > > > > > Research Center for Data-Driven Business & Big Data Analytics
> > > > > > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > > > > > FN 199 685 f, Commercial register court: LGZ Graz
> > > > > > >
> > > > > > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > > > > > mdokter@know-center.at
> > > > > > >
> > > > > > >   > AILEVATOR: Accelerating AI projects to the highest level:
> > > > > > > www.know-center.at/ailevator
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > > Senior Researcher
> > > >
> > > > Know-Center GmbH
> > > > Research Center for Data-Driven Business & Big Data Analytics
> > > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > > FN 199 685 f, Commercial register court: LGZ Graz
> > > >
> > > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > > mdokter@know-center.at
> > > >
> > > >  > AILEVATOR: Accelerating AI projects to the highest level:
> > > > www.know-center.at/ailevator
> > > >
> > >
>
>

-- 

Best,
Badrul

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Sebastian Baunsgaard <se...@bgaard.dk>.
Hi all, 

I hate to stop the release process, but we need to update dependencies with
critical vulnerabilities before we can release anything.
This include amoung other things Spark and Hadoop.

https://issues.apache.org/jira/browse/SYSTEMDS-3656
https://issues.apache.org/jira/browse/SYSTEMDS-3655

I do not have time to update these currently, because of other pressing issues.

Best regards
Sebastian

On Mon, 2023-12-04 at 09:10 +0530, Janardhan wrote:
> Hi all
> 
> A tentative code freeze date is 8th December based on the current jiras and
> PR queue.
> 
> If any of your PRs or bugs needs attention, please post a note here.
> 
> Make sure to include your contributions in the release iterations. If the
> functionality or performance is not as much as you expect - if the code
> reasonably serves the purpose it is important to merge into the team
> codebase.
> 
> It is ok, to merge your contributions incrementally in a few commits.
> 
> Code review and merge are crucial since the reviewer learns your
> implementation and perhaps add design inputs, assist in maintenance.
> 
> 
> Best regards
> Janardhan
> 
> 
> On Thu, Nov 30, 2023, 21:05 Janardhan <ja...@apache.org> wrote:
> 
> > Hi all
> > 
> > We are planning for a code freeze at the end of December first week.
> > 
> > Let us some time to document missing functions, & mark the jira resolved
> > if they are done.
> > 
> > If you need assistance or help for review please post a note here.
> > 
> > Best regards
> > Janardhan
> > 
> > On Mon, Nov 20, 2023, 20:05 Mark Dokter <md...@know-center.at.invalid>
> > wrote:
> > 
> > > Hi!
> > > 
> > > Waiting for a major release is fine by me - I'm not dependent on
> > > releases in my work ;-)
> > > 
> > > I'll push the changes to a branch on the systemds repo once I have it
> > > cleaned up and rebased on main.
> > > 
> > > Cheers,
> > > Mark
> > > 
> > > On 17.11.23 04:21, Janardhan wrote:
> > > > Hi Mark
> > > > 
> > > > Code freeze: Hoping for the next one/two weeks depending if there are
> > > > issues with the features already introduced.
> > > > 
> > > > Can we create a new branch and merge changes? Why not go ahead and
> > > create a
> > > > new branch in the team repo and merge changes. Let us give some airtime,
> > > > for the feature to make sure it is working as expected & no regressions
> > > > would be there.
> > > > 
> > > > 
> > > > We can do tests on that branch, later we can go for a major release.
> > > > 
> > > > 
> > > > Best Regards
> > > > Janardhan
> > > > 
> > > > 
> > > > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
> > > > 
> > > > > CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
> > > > > release to include CUDA 12.0 support.
> > > > > In that case, either we make this release a major release, or push
> > > > > the version upgrade to the next.
> > > > > 
> > > > > Regards,
> > > > > Arnab..
> > > > > 
> > > > > On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
> > > > > .invalid>
> > > > > wrote:
> > > > > 
> > > > > > Hi!
> > > > > > 
> > > > > > What's the time frame until release? I have CUDA 12.0 support working
> > > > > > but that needs cleanup and testing.
> > > > > > 
> > > > > > Regards, Mark
> > > > > > 
> > > > > > On 16.11.23 09:34, Matthias Boehm wrote:
> > > > > > > great - thanks for starting the process Janardhan. @all: we should
> > > all
> > > > > > > help close some of the old JIRA tasks and fix remaining known issues.
> > > > > > > 
> > > > > > > Regards,
> > > > > > > Matthias
> > > > > > > 
> > > > > > > On 11/16/2023 3:35 AM, Janardhan wrote:
> > > > > > > > Hi all,
> > > > > > > > 
> > > > > > > > Let us initiate the release preparations. I'd like to mention the
> > > > > > > > release date shortly after our discussion in this mail thread.
> > > > > > > > 
> > > > > > > > Let us take this opportunity to wrap up any features recently
> > > > > > > > introduced. Hopefully, the features are functional and reasonably
> > > > > > > > tested for the release. If not, you can mention here if you would
> > > need
> > > > > > > > some assistance, and reviews.
> > > > > > > > 
> > > > > > > > You can start with Updating version details in Jira as follows:
> > > > > > > > 
> > > > > > > > 1. Apply version
> > > > > > > >       Affects Version/s: SystemDS 3.2
> > > > > > > >       Fix Version/s: SystemDS 3.2
> > > > > > > > 
> > > > > > > > 2. Fill in Components the Jira is related to
> > > > > > > >       Component/s:
> > > > > > > > 
> > > > > > > > some of component names:
> > > > > > > > Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> > > > > > > > Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> > > > > > > > 
> > > > > > > > --
> > > > > > > > Helpful links
> > > > > > > > 
> > > > > > > > [1] all Jira components list:
> > > > > > > > 
> > > > > > 
> > > > > 
> > > https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> > > > > > > > [2] SystemDS Kanban Board:
> > > > > > > > 
> > > > > > 
> > > > > 
> > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> > > > > > > > [3] Release Process:
> > > > > > > > https://apache.github.io/systemds/site/release-process.html
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Best Regards,
> > > > > > > > Janardhan
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > --
> > > > > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > > > > Senior Researcher
> > > > > > 
> > > > > > Know-Center GmbH
> > > > > > Research Center for Data-Driven Business & Big Data Analytics
> > > > > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > > > > FN 199 685 f, Commercial register court: LGZ Graz
> > > > > > 
> > > > > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > > > > mdokter@know-center.at
> > > > > > 
> > > > > >   > AILEVATOR: Accelerating AI projects to the highest level:
> > > > > > www.know-center.at/ailevator
> > > > > > 
> > > > > 
> > > > 
> > > 
> > > 
> > > 
> > > --
> > > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > > Senior Researcher
> > > 
> > > Know-Center GmbH
> > > Research Center for Data-Driven Business & Big Data Analytics
> > > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > > FN 199 685 f, Commercial register court: LGZ Graz
> > > 
> > > Phone +43-316-873-30804 Mobile +43-664-88796966
> > > mdokter@know-center.at
> > > 
> > >  > AILEVATOR: Accelerating AI projects to the highest level:
> > > www.know-center.at/ailevator
> > > 
> >

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <ja...@apache.org>.
Hi all

A tentative code freeze date is 8th December based on the current jiras and
PR queue.

If any of your PRs or bugs needs attention, please post a note here.

Make sure to include your contributions in the release iterations. If the
functionality or performance is not as much as you expect - if the code
reasonably serves the purpose it is important to merge into the team
codebase.

It is ok, to merge your contributions incrementally in a few commits.

Code review and merge are crucial since the reviewer learns your
implementation and perhaps add design inputs, assist in maintenance.


Best regards
Janardhan


On Thu, Nov 30, 2023, 21:05 Janardhan <ja...@apache.org> wrote:

> Hi all
>
> We are planning for a code freeze at the end of December first week.
>
> Let us some time to document missing functions, & mark the jira resolved
> if they are done.
>
> If you need assistance or help for review please post a note here.
>
> Best regards
> Janardhan
>
> On Mon, Nov 20, 2023, 20:05 Mark Dokter <md...@know-center.at.invalid>
> wrote:
>
>> Hi!
>>
>> Waiting for a major release is fine by me - I'm not dependent on
>> releases in my work ;-)
>>
>> I'll push the changes to a branch on the systemds repo once I have it
>> cleaned up and rebased on main.
>>
>> Cheers,
>> Mark
>>
>> On 17.11.23 04:21, Janardhan wrote:
>> > Hi Mark
>> >
>> > Code freeze: Hoping for the next one/two weeks depending if there are
>> > issues with the features already introduced.
>> >
>> > Can we create a new branch and merge changes? Why not go ahead and
>> create a
>> > new branch in the team repo and merge changes. Let us give some airtime,
>> > for the feature to make sure it is working as expected & no regressions
>> > would be there.
>> >
>> >
>> > We can do tests on that branch, later we can go for a major release.
>> >
>> >
>> > Best Regards
>> > Janardhan
>> >
>> >
>> > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
>> >
>> >> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
>> >> release to include CUDA 12.0 support.
>> >> In that case, either we make this release a major release, or push
>> >> the version upgrade to the next.
>> >>
>> >> Regards,
>> >> Arnab..
>> >>
>> >> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
>> >> .invalid>
>> >> wrote:
>> >>
>> >>> Hi!
>> >>>
>> >>> What's the time frame until release? I have CUDA 12.0 support working
>> >>> but that needs cleanup and testing.
>> >>>
>> >>> Regards, Mark
>> >>>
>> >>> On 16.11.23 09:34, Matthias Boehm wrote:
>> >>>> great - thanks for starting the process Janardhan. @all: we should
>> all
>> >>>> help close some of the old JIRA tasks and fix remaining known issues.
>> >>>>
>> >>>> Regards,
>> >>>> Matthias
>> >>>>
>> >>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>> >>>>> Hi all,
>> >>>>>
>> >>>>> Let us initiate the release preparations. I'd like to mention the
>> >>>>> release date shortly after our discussion in this mail thread.
>> >>>>>
>> >>>>> Let us take this opportunity to wrap up any features recently
>> >>>>> introduced. Hopefully, the features are functional and reasonably
>> >>>>> tested for the release. If not, you can mention here if you would
>> need
>> >>>>> some assistance, and reviews.
>> >>>>>
>> >>>>> You can start with Updating version details in Jira as follows:
>> >>>>>
>> >>>>> 1. Apply version
>> >>>>>       Affects Version/s: SystemDS 3.2
>> >>>>>       Fix Version/s: SystemDS 3.2
>> >>>>>
>> >>>>> 2. Fill in Components the Jira is related to
>> >>>>>       Component/s:
>> >>>>>
>> >>>>> some of component names:
>> >>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>> >>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>> >>>>>
>> >>>>> --
>> >>>>> Helpful links
>> >>>>>
>> >>>>> [1] all Jira components list:
>> >>>>>
>> >>>
>> >>
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>> >>>>> [2] SystemDS Kanban Board:
>> >>>>>
>> >>>
>> >>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>> >>>>> [3] Release Process:
>> >>>>> https://apache.github.io/systemds/site/release-process.html
>> >>>>>
>> >>>>>
>> >>>>> Best Regards,
>> >>>>> Janardhan
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> >>> Senior Researcher
>> >>>
>> >>> Know-Center GmbH
>> >>> Research Center for Data-Driven Business & Big Data Analytics
>> >>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> >>> FN 199 685 f, Commercial register court: LGZ Graz
>> >>>
>> >>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> >>> mdokter@know-center.at
>> >>>
>> >>>   > AILEVATOR: Accelerating AI projects to the highest level:
>> >>> www.know-center.at/ailevator
>> >>>
>> >>
>> >
>>
>>
>>
>> --
>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>> Senior Researcher
>>
>> Know-Center GmbH
>> Research Center for Data-Driven Business & Big Data Analytics
>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>> FN 199 685 f, Commercial register court: LGZ Graz
>>
>> Phone +43-316-873-30804 Mobile +43-664-88796966
>> mdokter@know-center.at
>>
>>  > AILEVATOR: Accelerating AI projects to the highest level:
>> www.know-center.at/ailevator
>>
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <ja...@apache.org>.
Hi all

We are planning for a code freeze at the end of December first week.

Let us some time to document missing functions, & mark the jira resolved if
they are done.

If you need assistance or help for review please post a note here.

Best regards
Janardhan

On Mon, Nov 20, 2023, 20:05 Mark Dokter <md...@know-center.at.invalid>
wrote:

> Hi!
>
> Waiting for a major release is fine by me - I'm not dependent on
> releases in my work ;-)
>
> I'll push the changes to a branch on the systemds repo once I have it
> cleaned up and rebased on main.
>
> Cheers,
> Mark
>
> On 17.11.23 04:21, Janardhan wrote:
> > Hi Mark
> >
> > Code freeze: Hoping for the next one/two weeks depending if there are
> > issues with the features already introduced.
> >
> > Can we create a new branch and merge changes? Why not go ahead and
> create a
> > new branch in the team repo and merge changes. Let us give some airtime,
> > for the feature to make sure it is working as expected & no regressions
> > would be there.
> >
> >
> > We can do tests on that branch, later we can go for a major release.
> >
> >
> > Best Regards
> > Janardhan
> >
> >
> > On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
> >
> >> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
> >> release to include CUDA 12.0 support.
> >> In that case, either we make this release a major release, or push
> >> the version upgrade to the next.
> >>
> >> Regards,
> >> Arnab..
> >>
> >> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
> >> .invalid>
> >> wrote:
> >>
> >>> Hi!
> >>>
> >>> What's the time frame until release? I have CUDA 12.0 support working
> >>> but that needs cleanup and testing.
> >>>
> >>> Regards, Mark
> >>>
> >>> On 16.11.23 09:34, Matthias Boehm wrote:
> >>>> great - thanks for starting the process Janardhan. @all: we should all
> >>>> help close some of the old JIRA tasks and fix remaining known issues.
> >>>>
> >>>> Regards,
> >>>> Matthias
> >>>>
> >>>> On 11/16/2023 3:35 AM, Janardhan wrote:
> >>>>> Hi all,
> >>>>>
> >>>>> Let us initiate the release preparations. I'd like to mention the
> >>>>> release date shortly after our discussion in this mail thread.
> >>>>>
> >>>>> Let us take this opportunity to wrap up any features recently
> >>>>> introduced. Hopefully, the features are functional and reasonably
> >>>>> tested for the release. If not, you can mention here if you would
> need
> >>>>> some assistance, and reviews.
> >>>>>
> >>>>> You can start with Updating version details in Jira as follows:
> >>>>>
> >>>>> 1. Apply version
> >>>>>       Affects Version/s: SystemDS 3.2
> >>>>>       Fix Version/s: SystemDS 3.2
> >>>>>
> >>>>> 2. Fill in Components the Jira is related to
> >>>>>       Component/s:
> >>>>>
> >>>>> some of component names:
> >>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> >>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> >>>>>
> >>>>> --
> >>>>> Helpful links
> >>>>>
> >>>>> [1] all Jira components list:
> >>>>>
> >>>
> >>
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> >>>>> [2] SystemDS Kanban Board:
> >>>>>
> >>>
> >>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> >>>>> [3] Release Process:
> >>>>> https://apache.github.io/systemds/site/release-process.html
> >>>>>
> >>>>>
> >>>>> Best Regards,
> >>>>> Janardhan
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> >>> Senior Researcher
> >>>
> >>> Know-Center GmbH
> >>> Research Center for Data-Driven Business & Big Data Analytics
> >>> Sandgasse 36, 4th floor, 8010 Graz, Austria
> >>> FN 199 685 f, Commercial register court: LGZ Graz
> >>>
> >>> Phone +43-316-873-30804 Mobile +43-664-88796966
> >>> mdokter@know-center.at
> >>>
> >>>   > AILEVATOR: Accelerating AI projects to the highest level:
> >>> www.know-center.at/ailevator
> >>>
> >>
> >
>
>
>
> --
> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> Senior Researcher
>
> Know-Center GmbH
> Research Center for Data-Driven Business & Big Data Analytics
> Sandgasse 36, 4th floor, 8010 Graz, Austria
> FN 199 685 f, Commercial register court: LGZ Graz
>
> Phone +43-316-873-30804 Mobile +43-664-88796966
> mdokter@know-center.at
>
>  > AILEVATOR: Accelerating AI projects to the highest level:
> www.know-center.at/ailevator
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Mark Dokter <md...@know-center.at.INVALID>.
Hi!

Waiting for a major release is fine by me - I'm not dependent on 
releases in my work ;-)

I'll push the changes to a branch on the systemds repo once I have it 
cleaned up and rebased on main.

Cheers,
Mark

On 17.11.23 04:21, Janardhan wrote:
> Hi Mark
> 
> Code freeze: Hoping for the next one/two weeks depending if there are
> issues with the features already introduced.
> 
> Can we create a new branch and merge changes? Why not go ahead and create a
> new branch in the team repo and merge changes. Let us give some airtime,
> for the feature to make sure it is working as expected & no regressions
> would be there.
> 
> 
> We can do tests on that branch, later we can go for a major release.
> 
> 
> Best Regards
> Janardhan
> 
> 
> On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:
> 
>> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
>> release to include CUDA 12.0 support.
>> In that case, either we make this release a major release, or push
>> the version upgrade to the next.
>>
>> Regards,
>> Arnab..
>>
>> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
>> .invalid>
>> wrote:
>>
>>> Hi!
>>>
>>> What's the time frame until release? I have CUDA 12.0 support working
>>> but that needs cleanup and testing.
>>>
>>> Regards, Mark
>>>
>>> On 16.11.23 09:34, Matthias Boehm wrote:
>>>> great - thanks for starting the process Janardhan. @all: we should all
>>>> help close some of the old JIRA tasks and fix remaining known issues.
>>>>
>>>> Regards,
>>>> Matthias
>>>>
>>>> On 11/16/2023 3:35 AM, Janardhan wrote:
>>>>> Hi all,
>>>>>
>>>>> Let us initiate the release preparations. I'd like to mention the
>>>>> release date shortly after our discussion in this mail thread.
>>>>>
>>>>> Let us take this opportunity to wrap up any features recently
>>>>> introduced. Hopefully, the features are functional and reasonably
>>>>> tested for the release. If not, you can mention here if you would need
>>>>> some assistance, and reviews.
>>>>>
>>>>> You can start with Updating version details in Jira as follows:
>>>>>
>>>>> 1. Apply version
>>>>>       Affects Version/s: SystemDS 3.2
>>>>>       Fix Version/s: SystemDS 3.2
>>>>>
>>>>> 2. Fill in Components the Jira is related to
>>>>>       Component/s:
>>>>>
>>>>> some of component names:
>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>>>>>
>>>>> --
>>>>> Helpful links
>>>>>
>>>>> [1] all Jira components list:
>>>>>
>>>
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>>>>> [2] SystemDS Kanban Board:
>>>>>
>>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>>>>> [3] Release Process:
>>>>> https://apache.github.io/systemds/site/release-process.html
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> Janardhan
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
>>> Senior Researcher
>>>
>>> Know-Center GmbH
>>> Research Center for Data-Driven Business & Big Data Analytics
>>> Sandgasse 36, 4th floor, 8010 Graz, Austria
>>> FN 199 685 f, Commercial register court: LGZ Graz
>>>
>>> Phone +43-316-873-30804 Mobile +43-664-88796966
>>> mdokter@know-center.at
>>>
>>>   > AILEVATOR: Accelerating AI projects to the highest level:
>>> www.know-center.at/ailevator
>>>
>>
> 



-- 
Dipl.-Ing. Dr.techn. Mark Dokter, BSc
Senior Researcher

Know-Center GmbH
Research Center for Data-Driven Business & Big Data Analytics
Sandgasse 36, 4th floor, 8010 Graz, Austria
FN 199 685 f, Commercial register court: LGZ Graz

Phone +43-316-873-30804 Mobile +43-664-88796966
mdokter@know-center.at

 > AILEVATOR: Accelerating AI projects to the highest level: 
www.know-center.at/ailevator

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Janardhan <ja...@apache.org>.
Hi Mark

Code freeze: Hoping for the next one/two weeks depending if there are
issues with the features already introduced.

Can we create a new branch and merge changes? Why not go ahead and create a
new branch in the team repo and merge changes. Let us give some airtime,
for the feature to make sure it is working as expected & no regressions
would be there.


We can do tests on that branch, later we can go for a major release.


Best Regards
Janardhan


On Thu, Nov 16, 2023, 20:06 arnab phani <ph...@gmail.com> wrote:

> CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
> release to include CUDA 12.0 support.
> In that case, either we make this release a major release, or push
> the version upgrade to the next.
>
> Regards,
> Arnab..
>
> On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <mdokter@know-center.at
> .invalid>
> wrote:
>
> > Hi!
> >
> > What's the time frame until release? I have CUDA 12.0 support working
> > but that needs cleanup and testing.
> >
> > Regards, Mark
> >
> > On 16.11.23 09:34, Matthias Boehm wrote:
> > > great - thanks for starting the process Janardhan. @all: we should all
> > > help close some of the old JIRA tasks and fix remaining known issues.
> > >
> > > Regards,
> > > Matthias
> > >
> > > On 11/16/2023 3:35 AM, Janardhan wrote:
> > >> Hi all,
> > >>
> > >> Let us initiate the release preparations. I'd like to mention the
> > >> release date shortly after our discussion in this mail thread.
> > >>
> > >> Let us take this opportunity to wrap up any features recently
> > >> introduced. Hopefully, the features are functional and reasonably
> > >> tested for the release. If not, you can mention here if you would need
> > >> some assistance, and reviews.
> > >>
> > >> You can start with Updating version details in Jira as follows:
> > >>
> > >> 1. Apply version
> > >>      Affects Version/s: SystemDS 3.2
> > >>      Fix Version/s: SystemDS 3.2
> > >>
> > >> 2. Fill in Components the Jira is related to
> > >>      Component/s:
> > >>
> > >> some of component names:
> > >> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> > >> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> > >>
> > >> --
> > >> Helpful links
> > >>
> > >> [1] all Jira components list:
> > >>
> >
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> > >> [2] SystemDS Kanban Board:
> > >>
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> > >> [3] Release Process:
> > >> https://apache.github.io/systemds/site/release-process.html
> > >>
> > >>
> > >> Best Regards,
> > >> Janardhan
> > >>
> > >
> >
> >
> >
> > --
> > Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> > Senior Researcher
> >
> > Know-Center GmbH
> > Research Center for Data-Driven Business & Big Data Analytics
> > Sandgasse 36, 4th floor, 8010 Graz, Austria
> > FN 199 685 f, Commercial register court: LGZ Graz
> >
> > Phone +43-316-873-30804 Mobile +43-664-88796966
> > mdokter@know-center.at
> >
> >  > AILEVATOR: Accelerating AI projects to the highest level:
> > www.know-center.at/ailevator
> >
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by arnab phani <ph...@gmail.com>.
CUDA 12.0 is not backward compatible with 10.0, i.e. we need a major
release to include CUDA 12.0 support.
In that case, either we make this release a major release, or push
the version upgrade to the next.

Regards,
Arnab..

On Thu, Nov 16, 2023 at 3:06 PM Mark Dokter <md...@know-center.at.invalid>
wrote:

> Hi!
>
> What's the time frame until release? I have CUDA 12.0 support working
> but that needs cleanup and testing.
>
> Regards, Mark
>
> On 16.11.23 09:34, Matthias Boehm wrote:
> > great - thanks for starting the process Janardhan. @all: we should all
> > help close some of the old JIRA tasks and fix remaining known issues.
> >
> > Regards,
> > Matthias
> >
> > On 11/16/2023 3:35 AM, Janardhan wrote:
> >> Hi all,
> >>
> >> Let us initiate the release preparations. I'd like to mention the
> >> release date shortly after our discussion in this mail thread.
> >>
> >> Let us take this opportunity to wrap up any features recently
> >> introduced. Hopefully, the features are functional and reasonably
> >> tested for the release. If not, you can mention here if you would need
> >> some assistance, and reviews.
> >>
> >> You can start with Updating version details in Jira as follows:
> >>
> >> 1. Apply version
> >>      Affects Version/s: SystemDS 3.2
> >>      Fix Version/s: SystemDS 3.2
> >>
> >> 2. Fill in Components the Jira is related to
> >>      Component/s:
> >>
> >> some of component names:
> >> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> >> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
> >>
> >> --
> >> Helpful links
> >>
> >> [1] all Jira components list:
> >>
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> >> [2] SystemDS Kanban Board:
> >>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> >> [3] Release Process:
> >> https://apache.github.io/systemds/site/release-process.html
> >>
> >>
> >> Best Regards,
> >> Janardhan
> >>
> >
>
>
>
> --
> Dipl.-Ing. Dr.techn. Mark Dokter, BSc
> Senior Researcher
>
> Know-Center GmbH
> Research Center for Data-Driven Business & Big Data Analytics
> Sandgasse 36, 4th floor, 8010 Graz, Austria
> FN 199 685 f, Commercial register court: LGZ Graz
>
> Phone +43-316-873-30804 Mobile +43-664-88796966
> mdokter@know-center.at
>
>  > AILEVATOR: Accelerating AI projects to the highest level:
> www.know-center.at/ailevator
>

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Mark Dokter <md...@know-center.at.INVALID>.
Hi!

What's the time frame until release? I have CUDA 12.0 support working 
but that needs cleanup and testing.

Regards, Mark

On 16.11.23 09:34, Matthias Boehm wrote:
> great - thanks for starting the process Janardhan. @all: we should all 
> help close some of the old JIRA tasks and fix remaining known issues.
> 
> Regards,
> Matthias
> 
> On 11/16/2023 3:35 AM, Janardhan wrote:
>> Hi all,
>>
>> Let us initiate the release preparations. I'd like to mention the
>> release date shortly after our discussion in this mail thread.
>>
>> Let us take this opportunity to wrap up any features recently
>> introduced. Hopefully, the features are functional and reasonably
>> tested for the release. If not, you can mention here if you would need
>> some assistance, and reviews.
>>
>> You can start with Updating version details in Jira as follows:
>>
>> 1. Apply version
>>      Affects Version/s: SystemDS 3.2
>>      Fix Version/s: SystemDS 3.2
>>
>> 2. Fill in Components the Jira is related to
>>      Component/s:
>>
>> some of component names:
>> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>>
>> -- 
>> Helpful links
>>
>> [1] all Jira components list:
>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
>> [2] SystemDS Kanban Board:
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
>> [3] Release Process:
>> https://apache.github.io/systemds/site/release-process.html
>>
>>
>> Best Regards,
>> Janardhan
>>
> 



-- 
Dipl.-Ing. Dr.techn. Mark Dokter, BSc
Senior Researcher

Know-Center GmbH
Research Center for Data-Driven Business & Big Data Analytics
Sandgasse 36, 4th floor, 8010 Graz, Austria
FN 199 685 f, Commercial register court: LGZ Graz

Phone +43-316-873-30804 Mobile +43-664-88796966
mdokter@know-center.at

 > AILEVATOR: Accelerating AI projects to the highest level: 
www.know-center.at/ailevator

Re: [DISCUSS] SystemDS 3.2.0 release

Posted by Matthias Boehm <mb...@gmail.com>.
great - thanks for starting the process Janardhan. @all: we should all 
help close some of the old JIRA tasks and fix remaining known issues.

Regards,
Matthias

On 11/16/2023 3:35 AM, Janardhan wrote:
> Hi all,
>
> Let us initiate the release preparations. I'd like to mention the
> release date shortly after our discussion in this mail thread.
>
> Let us take this opportunity to wrap up any features recently
> introduced. Hopefully, the features are functional and reasonably
> tested for the release. If not, you can mention here if you would need
> some assistance, and reviews.
>
> You can start with Updating version details in Jira as follows:
>
> 1. Apply version
>      Affects Version/s: SystemDS 3.2
>      Fix Version/s: SystemDS 3.2
>
> 2. Fill in Components the Jira is related to
>      Component/s:
>
> some of component names:
> Documentation, Runtime, Compiler, Build, APIs, Algorithms,
> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA
>
> --
> Helpful links
>
> [1] all Jira components list:
> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page
> [2] SystemDS Kanban Board:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466
> [3] Release Process:
> https://apache.github.io/systemds/site/release-process.html
>
>
> Best Regards,
> Janardhan
>