You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemml.apache.org by Luciano Resende <lu...@gmail.com> on 2016/10/15 19:27:03 UTC

[VOTE] Apache SystemML 0.11.0-incubating (RC3)

Please vote on releasing the following candidate as Apache SystemML version
0.11.0-incubating !

The vote is open for at least 72 hours and passes if a majority of at least
3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache SystemML 0.11.0-incubating
[ ] -1 Do not release this package because ...

To learn more about Apache SystemML, please see http://systemml.apache.org/

The tag to be voted on is v0.11.0-incubating-rc1
(1baebfde400134b3af6d373c254ee084a6d28cc3)

https://github.com/apache/incubator-systemml/tree/
1baebfde400134b3af6d373c254ee084a6d28cc3

The release artifacts can be found at :

https://dist.apache.org/repos/dist/dev/incubator/systemml/0.
11.0-incubating-rc3/

The maven release artifacts, including signatures, digests, etc. can be
found at:

https://repository.apache.org/content/repositories/orgapachesystemml-1009/


=====================================
== Apache Incubator release policy ==
=====================================
Please find below the guide to release management during incubation:
http://incubator.apache.org/guides/releasemanagement.html

=======================================
== How can I help test this release? ==
=======================================
If you are a SystemML user, you can help us test this release by taking an
existing Algorithm or workload and running on this release candidate, then
reporting any regressions.

================================================
== What justifies a -1 vote for this release? ==
================================================
-1 votes should only occur for significant stop-ship bugs or legal related
issues (e.g. wrong license, missing header files, etc). Minor bugs or
regressions should not block this release.


-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Acs S <ac...@yahoo.com.INVALID>.
So far all issues Glen has seen based on running unit test cases using Spark 1.6 are related to Test Case code itself, not base code. Probably we don't need to include that change at this hour for 0.11 unless there are any critical issues we need to get in.
-Arvind

      From: Niketan Pansare <np...@us.ibm.com>
 To: dev@systemml.incubator.apache.org 
 Sent: Tuesday, October 18, 2016 5:37 PM
 Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
   
Glenn: Would you prefer to have https://github.com/apache/incubator-systemml/pull/269 in 0.11 release ?

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar

Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor correction on the RC tag name (the actual tag hash is correct):

From: Luciano Resende <lu...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 10/17/2016 09:06 PM
Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Please note the minor correction on the RC tag name (the actual tag hash is
correct):

The tag to be voted on is v0.11.0-incubating-rc3 (
1baebfde400134b3af6d373c254ee084a6d28cc3)


And off course, my +1


On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <lu...@gmail.com>
wrote:

>
> Please vote on releasing the following candidate as Apache SystemML
> version 0.11.0-incubating !
>
> The vote is open for at least 72 hours and passes if a majority of at
> least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache SystemML, please see
> http://systemml.apache.org/
>
> The tag to be voted on is v0.11.0-incubating-rc1 (
> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>
> https://github.com/apache/incubator-systemml/tree/1baebfde40
> 0134b3af6d373c254ee084a6d28cc3
>
> The release artifacts can be found at :
>
> https://dist.apache.org/repos/dist/dev/incubator/systemml/0.
> 11.0-incubating-rc3/
>
> The maven release artifacts, including signatures, digests, etc. can be
> found at:
>
> https://repository.apache.org/content/repositories/orgapachesystemml-1009/
>
>
> =====================================
> == Apache Incubator release policy ==
> =====================================
> Please find below the guide to release management during incubation:
> http://incubator.apache.org/guides/releasemanagement.html
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a SystemML user, you can help us test this release by taking an
> existing Algorithm or workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> -1 votes should only occur for significant stop-ship bugs or legal
> related issues (e.g. wrong license, missing header files, etc). Minor bugs
> or regressions should not block this release.
>
>

-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Niketan Pansare <np...@us.ibm.com>.
+1

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar



From:	dusenberrymw@gmail.com
To:	dev@systemml.incubator.apache.org
Date:	10/21/2016 11:25 AM
Subject:	Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Okay I found out that the error I was encountering occurred due to passing
in a DataFrame with an explicit row index column ("__INDEX") that contain
incorrect row indices. Basically, I had taken a large DataFrame with the
row index column  and sampled from it, without updating the row indices.
Thus, I was effectively left with sparse row indices -- i.e. I may have had
rows 2, 18, 587, 398678, etc. The current DataFrame conversion code appears
to not yet be able to handle sparse row indices and thus threw an
exception. When I correctly re-indexed the sampled DataFrame with dense row
indices, everything worked as expected. Of course, our conversion code
automatically adds row indices to a given DataFrame during conversion if
the user does not supply them explicitly. However, it can save a bit of
time on repeated usage if it is done explicitly in one prior batch job.

I don't think this should block this release, and we should instead think
about this for the next release.  I've created SYSTEMML-1053 to track this
issue.

I'm running a few more tests, and then I'll respond today with a vote.

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 20, 2016, at 10:48 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
>
> Similar release-process steps executed successfully on Windows.
>
> Performance test suite for large data still running; reviewing of
available log files in-progress.
>
> Thanks,
> Glenn
>
> Nakul Jindal ---10/20/2016 02:43:06 PM---Basic sanity tests pasts on
MacOS following the process here: http://apache.github.io/incubator-syst
>
> From: Nakul Jindal <na...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 10/20/2016 02:43 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>
>
>
>
> Basic sanity tests pasts on MacOS following the process here:
>
http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute

>
> (The in-memory jar was removed by [SYSTEMML-741])
>
> +1
>
> Nakul Jindal
>
>
> On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:
>
> > Okay I've been testing the release candidate on a large-scale problem,
and
> > I'm currently running into a "java.lang.NegativeArraySizeException" in
> > the SparseBlockMCSR that I do not believe was present previously. I'm
> > currently investigating, and will post again soon.
> >
> > On another note, I successfully ran all of the Python tests on both
Python
> > 2.7 and 3.5.
> >
> > -Mike
> >
> > --
> >
> > Mike Dusenberry
> > GitHub: github.com/dusenberrymw
> > LinkedIn: linkedin.com/in/mikedusenberry
> >
> > Sent from my iPhone.
> >
> >
> > > On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com>
wrote:
> > >
> > > Yes - that is correct for test cases involving ID column for
> > DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
> > MLContextTest. The four failures for MLContextFrameTest were slightly
> > different and involve similar fix as done for FrameConverterTest under
> > [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
> > incorporate schema information when converting to JavaRDD<Row>.
> > >
> > > Thanks,
> > > Glenn
> > >
> > > Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues
were
> > only caused by wrong tests that used an invalid ID schema or populated
> > >
> > > From: Matthias Boehm <mb...@googlemail.com>
> > > To: dev@systemml.incubator.apache.org
> > > Date: 10/19/2016 12:36 PM
> > > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >
> > >
> > >
> > >
> > > Glenn, all these issues were only caused by wrong tests that used an
> > > invalid ID schema or populated this column incorrectly, right? If so,
> > > then I think it's fine to release. However, if we touch it anyway, we
> > > should globally change the ID schema from double to long, which is
more
> > > intuitive when created by hand.
> > >
> > > Regards,
> > > Matthias
> > >
> > > On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> > > > OK, so I think it's my understanding that for the 'src' release for
> > rc3,
> > > > the pom is using Spark 1.4 and the test suite passes for Spark 1.4,
so
> > this
> > > > issue being discussed regarding test cases on Spark 1.6 is not a
> > blocker
> > > > for this release since the 'src' release builds and all tests pass.
> > > >
> > > > If this is not correct, could someone please correct me?
> > > >
> > > > Deron
> > > >
> > > >
> > > > On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
> > luckbr1975@gmail.com>
> > > > wrote:
> > > >
> > > >> if tests are consistently failing, then we should cancel the RC
and
> > either
> > > >> fix the test or mark it as @ignored.
> > > >>
> > > >> Intermittent fails might be ok, but it's a community decision.
> > > >>
> > > >> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
> > deroneriksson@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> I believe that for an Apache release, our test suite is supposed
to
> > pass
> > > >>> (although I'm pretty sure random test fails can be ignored).
> > > >>>
> > > >>> See 2.1 of Release Check List here:
> > > >>>
http://incubator.apache.org/guides/releasemanagement.html#check-list
> > > >>>
> > > >>> "2.1 Build is successful including automated tests.
> > > >>> The expanded source archive is expected to build and pass tests."
> > > >>>
> > > >>> Luciano, do you happen to know if some test failures are
acceptable
> > since
> > > >>> our test suite is so enormous (6300+ tests)?
> > > >>>
> > > >>> Deron
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner
<gw...@us.ibm.com>
> > > >>> wrote:
> > > >>>
> > > >>>> It's a nice-to-have but not a release blocker.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Glenn
> > > >>>>
> > > >>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
> > 05:38:26
> > > >>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
> > > >> Niketan
> > > >>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to
have
> > > >>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
> > releas
> > > >>>>
> > > >>>> From: Niketan Pansare/Almaden/IBM@IBMUS
> > > >>>> To: dev@systemml.incubator.apache.org
> > > >>>> Date: 10/18/2016 05:38 PM
> > > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > >>>> ------------------------------
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Glenn: Would you prefer to have
> > > >>>> *https://github.com/apache/incubator-systemml/pull/269*
> > > >>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> > > >> release
> > > >>> ?
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Niketan Pansare
> > > >>>> IBM Almaden Research Center
> > > >>>> E-mail: npansar At us.ibm.com
> > > >>>> *http://researcher.watson.ibm.com/researcher/view.php?
> > > >> person=us-npansar*
> > > >>>> <http://researcher.watson.ibm.com/researcher/view.php?
> > > >> person=us-npansar>
> > > >>>>
> > > >>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the
minor
> > > >>>> correction on the RC tag name (the actual tag hash is correct):
> > > >>>>
> > > >>>> From: Luciano Resende <lu...@gmail.com>
> > > >>>> To: dev@systemml.incubator.apache.org
> > > >>>> Date: 10/17/2016 09:06 PM
> > > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > >>>> ------------------------------
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Please note the minor correction on the RC tag name (the actual
tag
> > > >> hash
> > > >>> is
> > > >>>> correct):
> > > >>>>
> > > >>>> The tag to be voted on is v0.11.0-incubating-rc3 (
> > > >>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >>>>
> > > >>>>
> > > >>>> And off course, my +1
> > > >>>>
> > > >>>>
> > > >>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> > > >> luckbr1975@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>>
> > > >>>>> Please vote on releasing the following candidate as Apache
SystemML
> > > >>>>> version 0.11.0-incubating !
> > > >>>>>
> > > >>>>> The vote is open for at least 72 hours and passes if a majority
of
> > at
> > > >>>>> least 3 +1 PMC votes are cast.
> > > >>>>>
> > > >>>>> [ ] +1 Release this package as Apache SystemML
0.11.0-incubating
> > > >>>>> [ ] -1 Do not release this package because ...
> > > >>>>>
> > > >>>>> To learn more about Apache SystemML, please see
> > > >>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
> > > >>>>>
> > > >>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
> > > >>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >>>>>
> > > >>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > > >>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > > >>>>> 0134b3af6d373c254ee084a6d28cc3
> > > >>>>>
> > > >>>>> The release artifacts can be found at :
> > > >>>>>
> > > >>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > > >>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > > >>>>> 11.0-incubating-rc3/
> > > >>>>>
> > > >>>>> The maven release artifacts, including signatures, digests,
etc.
> > can
> > > >> be
> > > >>>>> found at:
> > > >>>>>
> > > >>>>>
> > > >>>> *https://repository.apache.org/content/repositories/
> > > >>> orgapachesystemml-1009/*
> > > >>>> <https://repository.apache.org/content/repositories/
> > > >>> orgapachesystemml-1009/>
> > > >>>>>
> > > >>>>>
> > > >>>>> =====================================
> > > >>>>> == Apache Incubator release policy ==
> > > >>>>> =====================================
> > > >>>>> Please find below the guide to release management during
> > incubation:
> > > >>>>> *http://incubator.apache.org/guides/releasemanagement.html*
> > > >>>> <http://incubator.apache.org/guides/releasemanagement.html>
> > > >>>>>
> > > >>>>> =======================================
> > > >>>>> == How can I help test this release? ==
> > > >>>>> =======================================
> > > >>>>> If you are a SystemML user, you can help us test this release
by
> > > >> taking
> > > >>>> an
> > > >>>>> existing Algorithm or workload and running on this release
> > candidate,
> > > >>>> then
> > > >>>>> reporting any regressions.
> > > >>>>>
> > > >>>>> ================================================
> > > >>>>> == What justifies a -1 vote for this release? ==
> > > >>>>> ================================================
> > > >>>>> -1 votes should only occur for significant stop-ship bugs or
legal
> > > >>>>> related issues (e.g. wrong license, missing header files, etc).
> > Minor
> > > >>>> bugs
> > > >>>>> or regressions should not block this release.
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>> Luciano Resende
> > > >>>> *http://twitter.com/lresende1975* <
http://twitter.com/lresende1975>
> > > >>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Luciano Resende
> > > >> http://twitter.com/lresende1975
> > > >> http://lresende.blogspot.com/
> > > >>
> > > >
> > >
> > >
> > >
> > >
> >
>
>
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Acs S <ac...@yahoo.com.INVALID>.
I have delivered fix for Jira SYSTEMML-1056 issue early morning. Berthold has verified it in his environment on reasonably large data as well..
We are ready to create 0.11 RC4.
Luciano, Can you please create RC4 soon? 
ThanksArvind

      From: "dusenberrymw@gmail.com" <du...@gmail.com>
 To: dev@systemml.incubator.apache.org 
 Sent: Saturday, October 22, 2016 10:13 AM
 Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
   
+1

I finished running some test jobs in my large scale scenario on this release candidate, and I think it is good to go.  Specifically, my scenario involved large numerical DataFrames, MLContext, matrices, DML, and multiple script invocations involving the various intermediate outputs.

One option would be to release this candidate as 0.11, and then follow up with a 0.11.1 release containing any bug fixes. This might make sense for edge-case bugs that don't impact normal usage.

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 22, 2016, at 2:14 AM, Berthold Reinwald <re...@us.ibm.com> wrote:
> 
> -1.
> 
> Transformencode throws an unnecessary error if strings to not comply with 
> the field requirements specified in RFC 4180. Arvind has a fix on the way 
> which should be included in the release. 
> 
> Regards,
> Berthold Reinwald
> IBM Almaden Research Center
> office: (408) 927 2208; T/L: 457 2208
> e-mail: reinwald@us.ibm.com
> 
> 
> 
> From:  dusenberrymw@gmail.com
> To:    dev@systemml.incubator.apache.org
> Date:  10/21/2016 11:25 AM
> Subject:        Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> 
> 
> 
> Okay I found out that the error I was encountering occurred due to passing 
> in a DataFrame with an explicit row index column ("__INDEX") that contain 
> incorrect row indices. Basically, I had taken a large DataFrame with the 
> row index column  and sampled from it, without updating the row indices. 
> Thus, I was effectively left with sparse row indices -- i.e. I may have 
> had rows 2, 18, 587, 398678, etc. The current DataFrame conversion code 
> appears to not yet be able to handle sparse row indices and thus threw an 
> exception. When I correctly re-indexed the sampled DataFrame with dense 
> row indices, everything worked as expected. Of course, our conversion code 
> automatically adds row indices to a given DataFrame during conversion if 
> the user does not supply them explicitly. However, it can save a bit of 
> time on repeated usage if it is done explicitly in one prior batch job.
> 
> I don't think this should block this release, and we should instead think 
> about this for the next release.  I've created SYSTEMML-1053 to track this 
> issue.
> 
> I'm running a few more tests, and then I'll respond today with a vote. 
> 
> -Mike
> 
> --
> 
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
> 
> Sent from my iPhone.
> 
> 
>> On Oct 20, 2016, at 10:48 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
>> 
>> Similar release-process steps executed successfully on Windows.
>> 
>> Performance test suite for large data still running; reviewing of 
> available log files in-progress.
>> 
>> Thanks,
>> Glenn
>> 
>> Nakul Jindal ---10/20/2016 02:43:06 PM---Basic sanity tests pasts on 
> MacOS following the process here: http://apache.github.io/incubator-syst
>> 
>> From: Nakul Jindal <na...@gmail.com>
>> To: dev@systemml.incubator.apache.org
>> Date: 10/20/2016 02:43 PM
>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>> 
>> 
>> 
>> 
>> Basic sanity tests pasts on MacOS following the process here:
>> 
> http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute
> 
>> 
>> (The in-memory jar was removed by [SYSTEMML-741])
>> 
>> +1
>> 
>> Nakul Jindal
>> 
>> 
>>> On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:
>>> 
>>> Okay I've been testing the release candidate on a large-scale problem, 
> and
>>> I'm currently running into a "java.lang.NegativeArraySizeException" in
>>> the SparseBlockMCSR that I do not believe was present previously. I'm
>>> currently investigating, and will post again soon.
>>> 
>>> On another note, I successfully ran all of the Python tests on both 
> Python
>>> 2.7 and 3.5.
>>> 
>>> -Mike
>>> 
>>> --
>>> 
>>> Mike Dusenberry
>>> GitHub: github.com/dusenberrymw
>>> LinkedIn: linkedin.com/in/mikedusenberry
>>> 
>>> Sent from my iPhone.
>>> 
>>> 
>>>> On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> 
> wrote:
>>>> 
>>>> Yes - that is correct for test cases involving ID column for
>>> DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
>>> MLContextTest. The four failures for MLContextFrameTest were slightly
>>> different and involve similar fix as done for FrameConverterTest under
>>> [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
>>> incorporate schema information when converting to JavaRDD<Row>.
>>>> 
>>>> Thanks,
>>>> Glenn
>>>> 
>>>> Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues 
> were
>>> only caused by wrong tests that used an invalid ID schema or populated
>>>> 
>>>> From: Matthias Boehm <mb...@googlemail.com>
>>>> To: dev@systemml.incubator.apache.org
>>>> Date: 10/19/2016 12:36 PM
>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Glenn, all these issues were only caused by wrong tests that used an
>>>> invalid ID schema or populated this column incorrectly, right? If 
> so,
>>>> then I think it's fine to release. However, if we touch it anyway, 
> we
>>>> should globally change the ID schema from double to long, which is 
> more
>>>> intuitive when created by hand.
>>>> 
>>>> Regards,
>>>> Matthias
>>>> 
>>>>> On 10/19/2016 8:30 PM, Deron Eriksson wrote:
>>>>> OK, so I think it's my understanding that for the 'src' release 
> for
>>> rc3,
>>>>> the pom is using Spark 1.4 and the test suite passes for Spark 
> 1.4, so
>>> this
>>>>> issue being discussed regarding test cases on Spark 1.6 is not a
>>> blocker
>>>>> for this release since the 'src' release builds and all tests 
> pass.
>>>>> 
>>>>> If this is not correct, could someone please correct me?
>>>>> 
>>>>> Deron
>>>>> 
>>>>> 
>>>>> On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
>>> luckbr1975@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> if tests are consistently failing, then we should cancel the RC 
> and
>>> either
>>>>>> fix the test or mark it as @ignored.
>>>>>> 
>>>>>> Intermittent fails might be ok, but it's a community decision.
>>>>>> 
>>>>>> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
>>> deroneriksson@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> I believe that for an Apache release, our test suite is supposed 
> to
>>> pass
>>>>>>> (although I'm pretty sure random test fails can be ignored).
>>>>>>> 
>>>>>>> See 2.1 of Release Check List here:
>>>>>>> 
> http://incubator.apache.org/guides/releasemanagement.html#check-list
>>>>>>> 
>>>>>>> "2.1 Build is successful including automated tests.
>>>>>>> The expanded source archive is expected to build and pass 
> tests."
>>>>>>> 
>>>>>>> Luciano, do you happen to know if some test failures are 
> acceptable
>>> since
>>>>>>> our test suite is so enormous (6300+ tests)?
>>>>>>> 
>>>>>>> Deron
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner 
> <gw...@us.ibm.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> It's a nice-to-have but not a release blocker.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Glenn
>>>>>>>> 
>>>>>>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
>>> 05:38:26
>>>>>>>> PM---Glenn: Would you prefer to have https://github.com/apache/
> ]
>>>>>> Niketan
>>>>>>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to 
> have
>>>>>>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
>>> releas
>>>>>>>> 
>>>>>>>> From: Niketan Pansare/Almaden/IBM@IBMUS
>>>>>>>> To: dev@systemml.incubator.apache.org
>>>>>>>> Date: 10/18/2016 05:38 PM
>>>>>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>>>>>> ------------------------------
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Glenn: Would you prefer to have
>>>>>>>> *https://github.com/apache/incubator-systemml/pull/269*
>>>>>>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
>>>>>> release
>>>>>>> ?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> 
>>>>>>>> Niketan Pansare
>>>>>>>> IBM Almaden Research Center
>>>>>>>> E-mail: npansar At us.ibm.com
>>>>>>>> *http://researcher.watson.ibm.com/researcher/view.php?
>>>>>> person=us-npansar*
>>>>>>>> <http://researcher.watson.ibm.com/researcher/view.php?
>>>>>> person=us-npansar>
>>>>>>>> 
>>>>>>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the 
> minor
>>>>>>>> correction on the RC tag name (the actual tag hash is correct):
>>>>>>>> 
>>>>>>>> From: Luciano Resende <lu...@gmail.com>
>>>>>>>> To: dev@systemml.incubator.apache.org
>>>>>>>> Date: 10/17/2016 09:06 PM
>>>>>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>>>>>> ------------------------------
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Please note the minor correction on the RC tag name (the actual 
> tag
>>>>>> hash
>>>>>>> is
>>>>>>>> correct):
>>>>>>>> 
>>>>>>>> The tag to be voted on is v0.11.0-incubating-rc3 (
>>>>>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> And off course, my +1
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
>>>>>> luckbr1975@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Please vote on releasing the following candidate as Apache 
> SystemML
>>>>>>>>> version 0.11.0-incubating !
>>>>>>>>> 
>>>>>>>>> The vote is open for at least 72 hours and passes if a 
> majority of
>>> at
>>>>>>>>> least 3 +1 PMC votes are cast.
>>>>>>>>> 
>>>>>>>>> [ ] +1 Release this package as Apache SystemML 
> 0.11.0-incubating
>>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>> 
>>>>>>>>> To learn more about Apache SystemML, please see
>>>>>>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
>>>>>>>>> 
>>>>>>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
>>>>>>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>>>>>> 
>>>>>>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
>>>>>>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
>>>>>>>>> 0134b3af6d373c254ee084a6d28cc3
>>>>>>>>> 
>>>>>>>>> The release artifacts can be found at :
>>>>>>>>> 
>>>>>>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
>>>>>>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
>>>>>>>>> 11.0-incubating-rc3/
>>>>>>>>> 
>>>>>>>>> The maven release artifacts, including signatures, digests, 
> etc.
>>> can
>>>>>> be
>>>>>>>>> found at:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> *https://repository.apache.org/content/repositories/
>>>>>>> orgapachesystemml-1009/*
>>>>>>>> <https://repository.apache.org/content/repositories/
>>>>>>> orgapachesystemml-1009/>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> =====================================
>>>>>>>>> == Apache Incubator release policy ==
>>>>>>>>> =====================================
>>>>>>>>> Please find below the guide to release management during
>>> incubation:
>>>>>>>>> *http://incubator.apache.org/guides/releasemanagement.html*
>>>>>>>> <http://incubator.apache.org/guides/releasemanagement.html>
>>>>>>>>> 
>>>>>>>>> =======================================
>>>>>>>>> == How can I help test this release? ==
>>>>>>>>> =======================================
>>>>>>>>> If you are a SystemML user, you can help us test this release 
> by
>>>>>> taking
>>>>>>>> an
>>>>>>>>> existing Algorithm or workload and running on this release
>>> candidate,
>>>>>>>> then
>>>>>>>>> reporting any regressions.
>>>>>>>>> 
>>>>>>>>> ================================================
>>>>>>>>> == What justifies a -1 vote for this release? ==
>>>>>>>>> ================================================
>>>>>>>>> -1 votes should only occur for significant stop-ship bugs or 
> legal
>>>>>>>>> related issues (e.g. wrong license, missing header files, 
> etc).
>>> Minor
>>>>>>>> bugs
>>>>>>>>> or regressions should not block this release.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Luciano Resende
>>>>>>>> *http://twitter.com/lresende1975* <
> http://twitter.com/lresende1975>
>>>>>>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Luciano Resende
>>>>>> http://twitter.com/lresende1975
>>>>>> http://lresende.blogspot.com/
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>> 
> 
> 
> 
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by du...@gmail.com.
+1

I finished running some test jobs in my large scale scenario on this release candidate, and I think it is good to go.  Specifically, my scenario involved large numerical DataFrames, MLContext, matrices, DML, and multiple script invocations involving the various intermediate outputs.

One option would be to release this candidate as 0.11, and then follow up with a 0.11.1 release containing any bug fixes. This might make sense for edge-case bugs that don't impact normal usage.

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 22, 2016, at 2:14 AM, Berthold Reinwald <re...@us.ibm.com> wrote:
> 
> -1.
> 
> Transformencode throws an unnecessary error if strings to not comply with 
> the field requirements specified in RFC 4180. Arvind has a fix on the way 
> which should be included in the release. 
> 
> Regards,
> Berthold Reinwald
> IBM Almaden Research Center
> office: (408) 927 2208; T/L: 457 2208
> e-mail: reinwald@us.ibm.com
> 
> 
> 
> From:   dusenberrymw@gmail.com
> To:     dev@systemml.incubator.apache.org
> Date:   10/21/2016 11:25 AM
> Subject:        Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> 
> 
> 
> Okay I found out that the error I was encountering occurred due to passing 
> in a DataFrame with an explicit row index column ("__INDEX") that contain 
> incorrect row indices. Basically, I had taken a large DataFrame with the 
> row index column  and sampled from it, without updating the row indices. 
> Thus, I was effectively left with sparse row indices -- i.e. I may have 
> had rows 2, 18, 587, 398678, etc. The current DataFrame conversion code 
> appears to not yet be able to handle sparse row indices and thus threw an 
> exception. When I correctly re-indexed the sampled DataFrame with dense 
> row indices, everything worked as expected. Of course, our conversion code 
> automatically adds row indices to a given DataFrame during conversion if 
> the user does not supply them explicitly. However, it can save a bit of 
> time on repeated usage if it is done explicitly in one prior batch job.
> 
> I don't think this should block this release, and we should instead think 
> about this for the next release.  I've created SYSTEMML-1053 to track this 
> issue.
> 
> I'm running a few more tests, and then I'll respond today with a vote. 
> 
> -Mike
> 
> --
> 
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
> 
> Sent from my iPhone.
> 
> 
>> On Oct 20, 2016, at 10:48 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
>> 
>> Similar release-process steps executed successfully on Windows.
>> 
>> Performance test suite for large data still running; reviewing of 
> available log files in-progress.
>> 
>> Thanks,
>> Glenn
>> 
>> Nakul Jindal ---10/20/2016 02:43:06 PM---Basic sanity tests pasts on 
> MacOS following the process here: http://apache.github.io/incubator-syst
>> 
>> From: Nakul Jindal <na...@gmail.com>
>> To: dev@systemml.incubator.apache.org
>> Date: 10/20/2016 02:43 PM
>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>> 
>> 
>> 
>> 
>> Basic sanity tests pasts on MacOS following the process here:
>> 
> http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute
> 
>> 
>> (The in-memory jar was removed by [SYSTEMML-741])
>> 
>> +1
>> 
>> Nakul Jindal
>> 
>> 
>>> On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:
>>> 
>>> Okay I've been testing the release candidate on a large-scale problem, 
> and
>>> I'm currently running into a "java.lang.NegativeArraySizeException" in
>>> the SparseBlockMCSR that I do not believe was present previously. I'm
>>> currently investigating, and will post again soon.
>>> 
>>> On another note, I successfully ran all of the Python tests on both 
> Python
>>> 2.7 and 3.5.
>>> 
>>> -Mike
>>> 
>>> --
>>> 
>>> Mike Dusenberry
>>> GitHub: github.com/dusenberrymw
>>> LinkedIn: linkedin.com/in/mikedusenberry
>>> 
>>> Sent from my iPhone.
>>> 
>>> 
>>>> On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> 
> wrote:
>>>> 
>>>> Yes - that is correct for test cases involving ID column for
>>> DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
>>> MLContextTest. The four failures for MLContextFrameTest were slightly
>>> different and involve similar fix as done for FrameConverterTest under
>>> [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
>>> incorporate schema information when converting to JavaRDD<Row>.
>>>> 
>>>> Thanks,
>>>> Glenn
>>>> 
>>>> Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues 
> were
>>> only caused by wrong tests that used an invalid ID schema or populated
>>>> 
>>>> From: Matthias Boehm <mb...@googlemail.com>
>>>> To: dev@systemml.incubator.apache.org
>>>> Date: 10/19/2016 12:36 PM
>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Glenn, all these issues were only caused by wrong tests that used an
>>>> invalid ID schema or populated this column incorrectly, right? If 
> so,
>>>> then I think it's fine to release. However, if we touch it anyway, 
> we
>>>> should globally change the ID schema from double to long, which is 
> more
>>>> intuitive when created by hand.
>>>> 
>>>> Regards,
>>>> Matthias
>>>> 
>>>>> On 10/19/2016 8:30 PM, Deron Eriksson wrote:
>>>>> OK, so I think it's my understanding that for the 'src' release 
> for
>>> rc3,
>>>>> the pom is using Spark 1.4 and the test suite passes for Spark 
> 1.4, so
>>> this
>>>>> issue being discussed regarding test cases on Spark 1.6 is not a
>>> blocker
>>>>> for this release since the 'src' release builds and all tests 
> pass.
>>>>> 
>>>>> If this is not correct, could someone please correct me?
>>>>> 
>>>>> Deron
>>>>> 
>>>>> 
>>>>> On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
>>> luckbr1975@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> if tests are consistently failing, then we should cancel the RC 
> and
>>> either
>>>>>> fix the test or mark it as @ignored.
>>>>>> 
>>>>>> Intermittent fails might be ok, but it's a community decision.
>>>>>> 
>>>>>> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
>>> deroneriksson@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> I believe that for an Apache release, our test suite is supposed 
> to
>>> pass
>>>>>>> (although I'm pretty sure random test fails can be ignored).
>>>>>>> 
>>>>>>> See 2.1 of Release Check List here:
>>>>>>> 
> http://incubator.apache.org/guides/releasemanagement.html#check-list
>>>>>>> 
>>>>>>> "2.1 Build is successful including automated tests.
>>>>>>> The expanded source archive is expected to build and pass 
> tests."
>>>>>>> 
>>>>>>> Luciano, do you happen to know if some test failures are 
> acceptable
>>> since
>>>>>>> our test suite is so enormous (6300+ tests)?
>>>>>>> 
>>>>>>> Deron
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner 
> <gw...@us.ibm.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> It's a nice-to-have but not a release blocker.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Glenn
>>>>>>>> 
>>>>>>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
>>> 05:38:26
>>>>>>>> PM---Glenn: Would you prefer to have https://github.com/apache/
> ]
>>>>>> Niketan
>>>>>>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to 
> have
>>>>>>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
>>> releas
>>>>>>>> 
>>>>>>>> From: Niketan Pansare/Almaden/IBM@IBMUS
>>>>>>>> To: dev@systemml.incubator.apache.org
>>>>>>>> Date: 10/18/2016 05:38 PM
>>>>>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>>>>>> ------------------------------
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Glenn: Would you prefer to have
>>>>>>>> *https://github.com/apache/incubator-systemml/pull/269*
>>>>>>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
>>>>>> release
>>>>>>> ?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> 
>>>>>>>> Niketan Pansare
>>>>>>>> IBM Almaden Research Center
>>>>>>>> E-mail: npansar At us.ibm.com
>>>>>>>> *http://researcher.watson.ibm.com/researcher/view.php?
>>>>>> person=us-npansar*
>>>>>>>> <http://researcher.watson.ibm.com/researcher/view.php?
>>>>>> person=us-npansar>
>>>>>>>> 
>>>>>>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the 
> minor
>>>>>>>> correction on the RC tag name (the actual tag hash is correct):
>>>>>>>> 
>>>>>>>> From: Luciano Resende <lu...@gmail.com>
>>>>>>>> To: dev@systemml.incubator.apache.org
>>>>>>>> Date: 10/17/2016 09:06 PM
>>>>>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>>>>>> ------------------------------
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Please note the minor correction on the RC tag name (the actual 
> tag
>>>>>> hash
>>>>>>> is
>>>>>>>> correct):
>>>>>>>> 
>>>>>>>> The tag to be voted on is v0.11.0-incubating-rc3 (
>>>>>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> And off course, my +1
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
>>>>>> luckbr1975@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Please vote on releasing the following candidate as Apache 
> SystemML
>>>>>>>>> version 0.11.0-incubating !
>>>>>>>>> 
>>>>>>>>> The vote is open for at least 72 hours and passes if a 
> majority of
>>> at
>>>>>>>>> least 3 +1 PMC votes are cast.
>>>>>>>>> 
>>>>>>>>> [ ] +1 Release this package as Apache SystemML 
> 0.11.0-incubating
>>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>> 
>>>>>>>>> To learn more about Apache SystemML, please see
>>>>>>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
>>>>>>>>> 
>>>>>>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
>>>>>>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>>>>>> 
>>>>>>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
>>>>>>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
>>>>>>>>> 0134b3af6d373c254ee084a6d28cc3
>>>>>>>>> 
>>>>>>>>> The release artifacts can be found at :
>>>>>>>>> 
>>>>>>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
>>>>>>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
>>>>>>>>> 11.0-incubating-rc3/
>>>>>>>>> 
>>>>>>>>> The maven release artifacts, including signatures, digests, 
> etc.
>>> can
>>>>>> be
>>>>>>>>> found at:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> *https://repository.apache.org/content/repositories/
>>>>>>> orgapachesystemml-1009/*
>>>>>>>> <https://repository.apache.org/content/repositories/
>>>>>>> orgapachesystemml-1009/>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> =====================================
>>>>>>>>> == Apache Incubator release policy ==
>>>>>>>>> =====================================
>>>>>>>>> Please find below the guide to release management during
>>> incubation:
>>>>>>>>> *http://incubator.apache.org/guides/releasemanagement.html*
>>>>>>>> <http://incubator.apache.org/guides/releasemanagement.html>
>>>>>>>>> 
>>>>>>>>> =======================================
>>>>>>>>> == How can I help test this release? ==
>>>>>>>>> =======================================
>>>>>>>>> If you are a SystemML user, you can help us test this release 
> by
>>>>>> taking
>>>>>>>> an
>>>>>>>>> existing Algorithm or workload and running on this release
>>> candidate,
>>>>>>>> then
>>>>>>>>> reporting any regressions.
>>>>>>>>> 
>>>>>>>>> ================================================
>>>>>>>>> == What justifies a -1 vote for this release? ==
>>>>>>>>> ================================================
>>>>>>>>> -1 votes should only occur for significant stop-ship bugs or 
> legal
>>>>>>>>> related issues (e.g. wrong license, missing header files, 
> etc).
>>> Minor
>>>>>>>> bugs
>>>>>>>>> or regressions should not block this release.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Luciano Resende
>>>>>>>> *http://twitter.com/lresende1975* <
> http://twitter.com/lresende1975>
>>>>>>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Luciano Resende
>>>>>> http://twitter.com/lresende1975
>>>>>> http://lresende.blogspot.com/
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>> 
> 
> 
> 
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Berthold Reinwald <re...@us.ibm.com>.
-1.

Transformencode throws an unnecessary error if strings to not comply with 
the field requirements specified in RFC 4180. Arvind has a fix on the way 
which should be included in the release. 

Regards,
Berthold Reinwald
IBM Almaden Research Center
office: (408) 927 2208; T/L: 457 2208
e-mail: reinwald@us.ibm.com



From:   dusenberrymw@gmail.com
To:     dev@systemml.incubator.apache.org
Date:   10/21/2016 11:25 AM
Subject:        Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Okay I found out that the error I was encountering occurred due to passing 
in a DataFrame with an explicit row index column ("__INDEX") that contain 
incorrect row indices. Basically, I had taken a large DataFrame with the 
row index column  and sampled from it, without updating the row indices. 
Thus, I was effectively left with sparse row indices -- i.e. I may have 
had rows 2, 18, 587, 398678, etc. The current DataFrame conversion code 
appears to not yet be able to handle sparse row indices and thus threw an 
exception. When I correctly re-indexed the sampled DataFrame with dense 
row indices, everything worked as expected. Of course, our conversion code 
automatically adds row indices to a given DataFrame during conversion if 
the user does not supply them explicitly. However, it can save a bit of 
time on repeated usage if it is done explicitly in one prior batch job.

I don't think this should block this release, and we should instead think 
about this for the next release.  I've created SYSTEMML-1053 to track this 
issue.

I'm running a few more tests, and then I'll respond today with a vote. 

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 20, 2016, at 10:48 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
> 
> Similar release-process steps executed successfully on Windows.
> 
> Performance test suite for large data still running; reviewing of 
available log files in-progress.
> 
> Thanks,
> Glenn
> 
> Nakul Jindal ---10/20/2016 02:43:06 PM---Basic sanity tests pasts on 
MacOS following the process here: http://apache.github.io/incubator-syst
> 
> From: Nakul Jindal <na...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 10/20/2016 02:43 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> 
> 
> 
> 
> Basic sanity tests pasts on MacOS following the process here:
> 
http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute

> 
> (The in-memory jar was removed by [SYSTEMML-741])
> 
> +1
> 
> Nakul Jindal
> 
> 
> On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:
> 
> > Okay I've been testing the release candidate on a large-scale problem, 
and
> > I'm currently running into a "java.lang.NegativeArraySizeException" in
> > the SparseBlockMCSR that I do not believe was present previously. I'm
> > currently investigating, and will post again soon.
> >
> > On another note, I successfully ran all of the Python tests on both 
Python
> > 2.7 and 3.5.
> >
> > -Mike
> >
> > --
> >
> > Mike Dusenberry
> > GitHub: github.com/dusenberrymw
> > LinkedIn: linkedin.com/in/mikedusenberry
> >
> > Sent from my iPhone.
> >
> >
> > > On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> 
wrote:
> > >
> > > Yes - that is correct for test cases involving ID column for
> > DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
> > MLContextTest. The four failures for MLContextFrameTest were slightly
> > different and involve similar fix as done for FrameConverterTest under
> > [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
> > incorporate schema information when converting to JavaRDD<Row>.
> > >
> > > Thanks,
> > > Glenn
> > >
> > > Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues 
were
> > only caused by wrong tests that used an invalid ID schema or populated
> > >
> > > From: Matthias Boehm <mb...@googlemail.com>
> > > To: dev@systemml.incubator.apache.org
> > > Date: 10/19/2016 12:36 PM
> > > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >
> > >
> > >
> > >
> > > Glenn, all these issues were only caused by wrong tests that used an
> > > invalid ID schema or populated this column incorrectly, right? If 
so,
> > > then I think it's fine to release. However, if we touch it anyway, 
we
> > > should globally change the ID schema from double to long, which is 
more
> > > intuitive when created by hand.
> > >
> > > Regards,
> > > Matthias
> > >
> > > On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> > > > OK, so I think it's my understanding that for the 'src' release 
for
> > rc3,
> > > > the pom is using Spark 1.4 and the test suite passes for Spark 
1.4, so
> > this
> > > > issue being discussed regarding test cases on Spark 1.6 is not a
> > blocker
> > > > for this release since the 'src' release builds and all tests 
pass.
> > > >
> > > > If this is not correct, could someone please correct me?
> > > >
> > > > Deron
> > > >
> > > >
> > > > On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
> > luckbr1975@gmail.com>
> > > > wrote:
> > > >
> > > >> if tests are consistently failing, then we should cancel the RC 
and
> > either
> > > >> fix the test or mark it as @ignored.
> > > >>
> > > >> Intermittent fails might be ok, but it's a community decision.
> > > >>
> > > >> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
> > deroneriksson@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> I believe that for an Apache release, our test suite is supposed 
to
> > pass
> > > >>> (although I'm pretty sure random test fails can be ignored).
> > > >>>
> > > >>> See 2.1 of Release Check List here:
> > > >>> 
http://incubator.apache.org/guides/releasemanagement.html#check-list
> > > >>>
> > > >>> "2.1 Build is successful including automated tests.
> > > >>> The expanded source archive is expected to build and pass 
tests."
> > > >>>
> > > >>> Luciano, do you happen to know if some test failures are 
acceptable
> > since
> > > >>> our test suite is so enormous (6300+ tests)?
> > > >>>
> > > >>> Deron
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner 
<gw...@us.ibm.com>
> > > >>> wrote:
> > > >>>
> > > >>>> It's a nice-to-have but not a release blocker.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Glenn
> > > >>>>
> > > >>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
> > 05:38:26
> > > >>>> PM---Glenn: Would you prefer to have https://github.com/apache/
]
> > > >> Niketan
> > > >>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to 
have
> > > >>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
> > releas
> > > >>>>
> > > >>>> From: Niketan Pansare/Almaden/IBM@IBMUS
> > > >>>> To: dev@systemml.incubator.apache.org
> > > >>>> Date: 10/18/2016 05:38 PM
> > > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > >>>> ------------------------------
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Glenn: Would you prefer to have
> > > >>>> *https://github.com/apache/incubator-systemml/pull/269*
> > > >>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> > > >> release
> > > >>> ?
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Niketan Pansare
> > > >>>> IBM Almaden Research Center
> > > >>>> E-mail: npansar At us.ibm.com
> > > >>>> *http://researcher.watson.ibm.com/researcher/view.php?
> > > >> person=us-npansar*
> > > >>>> <http://researcher.watson.ibm.com/researcher/view.php?
> > > >> person=us-npansar>
> > > >>>>
> > > >>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the 
minor
> > > >>>> correction on the RC tag name (the actual tag hash is correct):
> > > >>>>
> > > >>>> From: Luciano Resende <lu...@gmail.com>
> > > >>>> To: dev@systemml.incubator.apache.org
> > > >>>> Date: 10/17/2016 09:06 PM
> > > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > >>>> ------------------------------
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Please note the minor correction on the RC tag name (the actual 
tag
> > > >> hash
> > > >>> is
> > > >>>> correct):
> > > >>>>
> > > >>>> The tag to be voted on is v0.11.0-incubating-rc3 (
> > > >>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >>>>
> > > >>>>
> > > >>>> And off course, my +1
> > > >>>>
> > > >>>>
> > > >>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> > > >> luckbr1975@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>>
> > > >>>>> Please vote on releasing the following candidate as Apache 
SystemML
> > > >>>>> version 0.11.0-incubating !
> > > >>>>>
> > > >>>>> The vote is open for at least 72 hours and passes if a 
majority of
> > at
> > > >>>>> least 3 +1 PMC votes are cast.
> > > >>>>>
> > > >>>>> [ ] +1 Release this package as Apache SystemML 
0.11.0-incubating
> > > >>>>> [ ] -1 Do not release this package because ...
> > > >>>>>
> > > >>>>> To learn more about Apache SystemML, please see
> > > >>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
> > > >>>>>
> > > >>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
> > > >>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >>>>>
> > > >>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > > >>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > > >>>>> 0134b3af6d373c254ee084a6d28cc3
> > > >>>>>
> > > >>>>> The release artifacts can be found at :
> > > >>>>>
> > > >>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > > >>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > > >>>>> 11.0-incubating-rc3/
> > > >>>>>
> > > >>>>> The maven release artifacts, including signatures, digests, 
etc.
> > can
> > > >> be
> > > >>>>> found at:
> > > >>>>>
> > > >>>>>
> > > >>>> *https://repository.apache.org/content/repositories/
> > > >>> orgapachesystemml-1009/*
> > > >>>> <https://repository.apache.org/content/repositories/
> > > >>> orgapachesystemml-1009/>
> > > >>>>>
> > > >>>>>
> > > >>>>> =====================================
> > > >>>>> == Apache Incubator release policy ==
> > > >>>>> =====================================
> > > >>>>> Please find below the guide to release management during
> > incubation:
> > > >>>>> *http://incubator.apache.org/guides/releasemanagement.html*
> > > >>>> <http://incubator.apache.org/guides/releasemanagement.html>
> > > >>>>>
> > > >>>>> =======================================
> > > >>>>> == How can I help test this release? ==
> > > >>>>> =======================================
> > > >>>>> If you are a SystemML user, you can help us test this release 
by
> > > >> taking
> > > >>>> an
> > > >>>>> existing Algorithm or workload and running on this release
> > candidate,
> > > >>>> then
> > > >>>>> reporting any regressions.
> > > >>>>>
> > > >>>>> ================================================
> > > >>>>> == What justifies a -1 vote for this release? ==
> > > >>>>> ================================================
> > > >>>>> -1 votes should only occur for significant stop-ship bugs or 
legal
> > > >>>>> related issues (e.g. wrong license, missing header files, 
etc).
> > Minor
> > > >>>> bugs
> > > >>>>> or regressions should not block this release.
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>> Luciano Resende
> > > >>>> *http://twitter.com/lresende1975* <
http://twitter.com/lresende1975>
> > > >>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Luciano Resende
> > > >> http://twitter.com/lresende1975
> > > >> http://lresende.blogspot.com/
> > > >>
> > > >
> > >
> > >
> > >
> > >
> >
> 
> 
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by du...@gmail.com.
Okay I found out that the error I was encountering occurred due to passing in a DataFrame with an explicit row index column ("__INDEX") that contain incorrect row indices. Basically, I had taken a large DataFrame with the row index column  and sampled from it, without updating the row indices. Thus, I was effectively left with sparse row indices -- i.e. I may have had rows 2, 18, 587, 398678, etc. The current DataFrame conversion code appears to not yet be able to handle sparse row indices and thus threw an exception. When I correctly re-indexed the sampled DataFrame with dense row indices, everything worked as expected. Of course, our conversion code automatically adds row indices to a given DataFrame during conversion if the user does not supply them explicitly. However, it can save a bit of time on repeated usage if it is done explicitly in one prior batch job.

I don't think this should block this release, and we should instead think about this for the next release.  I've created SYSTEMML-1053 to track this issue.

I'm running a few more tests, and then I'll respond today with a vote. 

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 20, 2016, at 10:48 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
> 
> Similar release-process steps executed successfully on Windows.
> 
> Performance test suite for large data still running; reviewing of available log files in-progress.
> 
> Thanks,
> Glenn
> 
> Nakul Jindal ---10/20/2016 02:43:06 PM---Basic sanity tests pasts on MacOS following the process here: http://apache.github.io/incubator-syst
> 
> From: Nakul Jindal <na...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 10/20/2016 02:43 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> 
> 
> 
> 
> Basic sanity tests pasts on MacOS following the process here:
> http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute
> 
> (The in-memory jar was removed by [SYSTEMML-741])
> 
> +1
> 
> Nakul Jindal
> 
> 
> On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:
> 
> > Okay I've been testing the release candidate on a large-scale problem, and
> > I'm currently running into a "java.lang.NegativeArraySizeException" in
> > the SparseBlockMCSR that I do not believe was present previously. I'm
> > currently investigating, and will post again soon.
> >
> > On another note, I successfully ran all of the Python tests on both Python
> > 2.7 and 3.5.
> >
> > -Mike
> >
> > --
> >
> > Mike Dusenberry
> > GitHub: github.com/dusenberrymw
> > LinkedIn: linkedin.com/in/mikedusenberry
> >
> > Sent from my iPhone.
> >
> >
> > > On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
> > >
> > > Yes - that is correct for test cases involving ID column for
> > DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
> > MLContextTest. The four failures for MLContextFrameTest were slightly
> > different and involve similar fix as done for FrameConverterTest under
> > [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
> > incorporate schema information when converting to JavaRDD<Row>.
> > >
> > > Thanks,
> > > Glenn
> > >
> > > Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues were
> > only caused by wrong tests that used an invalid ID schema or populated
> > >
> > > From: Matthias Boehm <mb...@googlemail.com>
> > > To: dev@systemml.incubator.apache.org
> > > Date: 10/19/2016 12:36 PM
> > > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >
> > >
> > >
> > >
> > > Glenn, all these issues were only caused by wrong tests that used an
> > > invalid ID schema or populated this column incorrectly, right? If so,
> > > then I think it's fine to release. However, if we touch it anyway, we
> > > should globally change the ID schema from double to long, which is more
> > > intuitive when created by hand.
> > >
> > > Regards,
> > > Matthias
> > >
> > > On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> > > > OK, so I think it's my understanding that for the 'src' release for
> > rc3,
> > > > the pom is using Spark 1.4 and the test suite passes for Spark 1.4, so
> > this
> > > > issue being discussed regarding test cases on Spark 1.6 is not a
> > blocker
> > > > for this release since the 'src' release builds and all tests pass.
> > > >
> > > > If this is not correct, could someone please correct me?
> > > >
> > > > Deron
> > > >
> > > >
> > > > On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
> > luckbr1975@gmail.com>
> > > > wrote:
> > > >
> > > >> if tests are consistently failing, then we should cancel the RC and
> > either
> > > >> fix the test or mark it as @ignored.
> > > >>
> > > >> Intermittent fails might be ok, but it's a community decision.
> > > >>
> > > >> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
> > deroneriksson@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> I believe that for an Apache release, our test suite is supposed to
> > pass
> > > >>> (although I'm pretty sure random test fails can be ignored).
> > > >>>
> > > >>> See 2.1 of Release Check List here:
> > > >>> http://incubator.apache.org/guides/releasemanagement.html#check-list
> > > >>>
> > > >>> "2.1 Build is successful including automated tests.
> > > >>> The expanded source archive is expected to build and pass tests."
> > > >>>
> > > >>> Luciano, do you happen to know if some test failures are acceptable
> > since
> > > >>> our test suite is so enormous (6300+ tests)?
> > > >>>
> > > >>> Deron
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
> > > >>> wrote:
> > > >>>
> > > >>>> It's a nice-to-have but not a release blocker.
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Glenn
> > > >>>>
> > > >>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
> > 05:38:26
> > > >>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
> > > >> Niketan
> > > >>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> > > >>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
> > releas
> > > >>>>
> > > >>>> From: Niketan Pansare/Almaden/IBM@IBMUS
> > > >>>> To: dev@systemml.incubator.apache.org
> > > >>>> Date: 10/18/2016 05:38 PM
> > > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > >>>> ------------------------------
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Glenn: Would you prefer to have
> > > >>>> *https://github.com/apache/incubator-systemml/pull/269*
> > > >>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> > > >> release
> > > >>> ?
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Niketan Pansare
> > > >>>> IBM Almaden Research Center
> > > >>>> E-mail: npansar At us.ibm.com
> > > >>>> *http://researcher.watson.ibm.com/researcher/view.php?
> > > >> person=us-npansar*
> > > >>>> <http://researcher.watson.ibm.com/researcher/view.php?
> > > >> person=us-npansar>
> > > >>>>
> > > >>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> > > >>>> correction on the RC tag name (the actual tag hash is correct):
> > > >>>>
> > > >>>> From: Luciano Resende <lu...@gmail.com>
> > > >>>> To: dev@systemml.incubator.apache.org
> > > >>>> Date: 10/17/2016 09:06 PM
> > > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > >>>> ------------------------------
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Please note the minor correction on the RC tag name (the actual tag
> > > >> hash
> > > >>> is
> > > >>>> correct):
> > > >>>>
> > > >>>> The tag to be voted on is v0.11.0-incubating-rc3 (
> > > >>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >>>>
> > > >>>>
> > > >>>> And off course, my +1
> > > >>>>
> > > >>>>
> > > >>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> > > >> luckbr1975@gmail.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>>
> > > >>>>> Please vote on releasing the following candidate as Apache SystemML
> > > >>>>> version 0.11.0-incubating !
> > > >>>>>
> > > >>>>> The vote is open for at least 72 hours and passes if a majority of
> > at
> > > >>>>> least 3 +1 PMC votes are cast.
> > > >>>>>
> > > >>>>> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> > > >>>>> [ ] -1 Do not release this package because ...
> > > >>>>>
> > > >>>>> To learn more about Apache SystemML, please see
> > > >>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
> > > >>>>>
> > > >>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
> > > >>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >>>>>
> > > >>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > > >>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > > >>>>> 0134b3af6d373c254ee084a6d28cc3
> > > >>>>>
> > > >>>>> The release artifacts can be found at :
> > > >>>>>
> > > >>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > > >>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > > >>>>> 11.0-incubating-rc3/
> > > >>>>>
> > > >>>>> The maven release artifacts, including signatures, digests, etc.
> > can
> > > >> be
> > > >>>>> found at:
> > > >>>>>
> > > >>>>>
> > > >>>> *https://repository.apache.org/content/repositories/
> > > >>> orgapachesystemml-1009/*
> > > >>>> <https://repository.apache.org/content/repositories/
> > > >>> orgapachesystemml-1009/>
> > > >>>>>
> > > >>>>>
> > > >>>>> =====================================
> > > >>>>> == Apache Incubator release policy ==
> > > >>>>> =====================================
> > > >>>>> Please find below the guide to release management during
> > incubation:
> > > >>>>> *http://incubator.apache.org/guides/releasemanagement.html*
> > > >>>> <http://incubator.apache.org/guides/releasemanagement.html>
> > > >>>>>
> > > >>>>> =======================================
> > > >>>>> == How can I help test this release? ==
> > > >>>>> =======================================
> > > >>>>> If you are a SystemML user, you can help us test this release by
> > > >> taking
> > > >>>> an
> > > >>>>> existing Algorithm or workload and running on this release
> > candidate,
> > > >>>> then
> > > >>>>> reporting any regressions.
> > > >>>>>
> > > >>>>> ================================================
> > > >>>>> == What justifies a -1 vote for this release? ==
> > > >>>>> ================================================
> > > >>>>> -1 votes should only occur for significant stop-ship bugs or legal
> > > >>>>> related issues (e.g. wrong license, missing header files, etc).
> > Minor
> > > >>>> bugs
> > > >>>>> or regressions should not block this release.
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>>> --
> > > >>>> Luciano Resende
> > > >>>> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
> > > >>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Luciano Resende
> > > >> http://twitter.com/lresende1975
> > > >> http://lresende.blogspot.com/
> > > >>
> > > >
> > >
> > >
> > >
> > >
> >
> 
> 
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Glenn Weidner <gw...@us.ibm.com>.
Similar release-process steps executed successfully on Windows.

Performance test suite for large data still running; reviewing of available
log files in-progress.

Thanks,
Glenn



From:	Nakul Jindal <na...@gmail.com>
To:	dev@systemml.incubator.apache.org
Date:	10/20/2016 02:43 PM
Subject:	Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Basic sanity tests pasts on MacOS following the process here:
http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute


(The in-memory jar was removed by [SYSTEMML-741])

+1

Nakul Jindal


On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:

> Okay I've been testing the release candidate on a large-scale problem,
and
> I'm currently running into a "java.lang.NegativeArraySizeException" in
> the SparseBlockMCSR that I do not believe was present previously. I'm
> currently investigating, and will post again soon.
>
> On another note, I successfully ran all of the Python tests on both
Python
> 2.7 and 3.5.
>
> -Mike
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
> >
> > Yes - that is correct for test cases involving ID column for
> DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
> MLContextTest. The four failures for MLContextFrameTest were slightly
> different and involve similar fix as done for FrameConverterTest under
> [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
> incorporate schema information when converting to JavaRDD<Row>.
> >
> > Thanks,
> > Glenn
> >
> > Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues were
> only caused by wrong tests that used an invalid ID schema or populated
> >
> > From: Matthias Boehm <mb...@googlemail.com>
> > To: dev@systemml.incubator.apache.org
> > Date: 10/19/2016 12:36 PM
> > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> >
> >
> >
> >
> > Glenn, all these issues were only caused by wrong tests that used an
> > invalid ID schema or populated this column incorrectly, right? If so,
> > then I think it's fine to release. However, if we touch it anyway, we
> > should globally change the ID schema from double to long, which is more
> > intuitive when created by hand.
> >
> > Regards,
> > Matthias
> >
> > On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> > > OK, so I think it's my understanding that for the 'src' release for
> rc3,
> > > the pom is using Spark 1.4 and the test suite passes for Spark 1.4,
so
> this
> > > issue being discussed regarding test cases on Spark 1.6 is not a
> blocker
> > > for this release since the 'src' release builds and all tests pass.
> > >
> > > If this is not correct, could someone please correct me?
> > >
> > > Deron
> > >
> > >
> > > On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
> luckbr1975@gmail.com>
> > > wrote:
> > >
> > >> if tests are consistently failing, then we should cancel the RC and
> either
> > >> fix the test or mark it as @ignored.
> > >>
> > >> Intermittent fails might be ok, but it's a community decision.
> > >>
> > >> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
> deroneriksson@gmail.com>
> > >> wrote:
> > >>
> > >>> I believe that for an Apache release, our test suite is supposed to
> pass
> > >>> (although I'm pretty sure random test fails can be ignored).
> > >>>
> > >>> See 2.1 of Release Check List here:
> > >>>
http://incubator.apache.org/guides/releasemanagement.html#check-list
> > >>>
> > >>> "2.1 Build is successful including automated tests.
> > >>> The expanded source archive is expected to build and pass tests."
> > >>>
> > >>> Luciano, do you happen to know if some test failures are acceptable
> since
> > >>> our test suite is so enormous (6300+ tests)?
> > >>>
> > >>> Deron
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner
<gw...@us.ibm.com>
> > >>> wrote:
> > >>>
> > >>>> It's a nice-to-have but not a release blocker.
> > >>>>
> > >>>> Thanks,
> > >>>> Glenn
> > >>>>
> > >>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
> 05:38:26
> > >>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
> > >> Niketan
> > >>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> > >>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
> releas
> > >>>>
> > >>>> From: Niketan Pansare/Almaden/IBM@IBMUS
> > >>>> To: dev@systemml.incubator.apache.org
> > >>>> Date: 10/18/2016 05:38 PM
> > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >>>> ------------------------------
> > >>>>
> > >>>>
> > >>>>
> > >>>> Glenn: Would you prefer to have
> > >>>> *https://github.com/apache/incubator-systemml/pull/269*
> > >>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> > >> release
> > >>> ?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Niketan Pansare
> > >>>> IBM Almaden Research Center
> > >>>> E-mail: npansar At us.ibm.com
> > >>>> *http://researcher.watson.ibm.com/researcher/view.php?
> > >> person=us-npansar*
> > >>>> <http://researcher.watson.ibm.com/researcher/view.php?
> > >> person=us-npansar>
> > >>>>
> > >>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> > >>>> correction on the RC tag name (the actual tag hash is correct):
> > >>>>
> > >>>> From: Luciano Resende <lu...@gmail.com>
> > >>>> To: dev@systemml.incubator.apache.org
> > >>>> Date: 10/17/2016 09:06 PM
> > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >>>> ------------------------------
> > >>>>
> > >>>>
> > >>>>
> > >>>> Please note the minor correction on the RC tag name (the actual
tag
> > >> hash
> > >>> is
> > >>>> correct):
> > >>>>
> > >>>> The tag to be voted on is v0.11.0-incubating-rc3 (
> > >>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > >>>>
> > >>>>
> > >>>> And off course, my +1
> > >>>>
> > >>>>
> > >>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> > >> luckbr1975@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>>
> > >>>>> Please vote on releasing the following candidate as Apache
SystemML
> > >>>>> version 0.11.0-incubating !
> > >>>>>
> > >>>>> The vote is open for at least 72 hours and passes if a majority
of
> at
> > >>>>> least 3 +1 PMC votes are cast.
> > >>>>>
> > >>>>> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> > >>>>> [ ] -1 Do not release this package because ...
> > >>>>>
> > >>>>> To learn more about Apache SystemML, please see
> > >>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
> > >>>>>
> > >>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
> > >>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > >>>>>
> > >>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > >>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > >>>>> 0134b3af6d373c254ee084a6d28cc3
> > >>>>>
> > >>>>> The release artifacts can be found at :
> > >>>>>
> > >>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > >>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > >>>>> 11.0-incubating-rc3/
> > >>>>>
> > >>>>> The maven release artifacts, including signatures, digests, etc.
> can
> > >> be
> > >>>>> found at:
> > >>>>>
> > >>>>>
> > >>>> *https://repository.apache.org/content/repositories/
> > >>> orgapachesystemml-1009/*
> > >>>> <https://repository.apache.org/content/repositories/
> > >>> orgapachesystemml-1009/>
> > >>>>>
> > >>>>>
> > >>>>> =====================================
> > >>>>> == Apache Incubator release policy ==
> > >>>>> =====================================
> > >>>>> Please find below the guide to release management during
> incubation:
> > >>>>> *http://incubator.apache.org/guides/releasemanagement.html*
> > >>>> <http://incubator.apache.org/guides/releasemanagement.html>
> > >>>>>
> > >>>>> =======================================
> > >>>>> == How can I help test this release? ==
> > >>>>> =======================================
> > >>>>> If you are a SystemML user, you can help us test this release by
> > >> taking
> > >>>> an
> > >>>>> existing Algorithm or workload and running on this release
> candidate,
> > >>>> then
> > >>>>> reporting any regressions.
> > >>>>>
> > >>>>> ================================================
> > >>>>> == What justifies a -1 vote for this release? ==
> > >>>>> ================================================
> > >>>>> -1 votes should only occur for significant stop-ship bugs or
legal
> > >>>>> related issues (e.g. wrong license, missing header files, etc).
> Minor
> > >>>> bugs
> > >>>>> or regressions should not block this release.
> > >>>>>
> > >>>>>
> > >>>>
> > >>>> --
> > >>>> Luciano Resende
> > >>>> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975
>
> > >>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Luciano Resende
> > >> http://twitter.com/lresende1975
> > >> http://lresende.blogspot.com/
> > >>
> > >
> >
> >
> >
> >
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Nakul Jindal <na...@gmail.com>.
Basic sanity tests pasts on MacOS following the process here:
http://apache.github.io/incubator-systemml/release-process.html#all-binaries-execute

(The in-memory jar was removed by [SYSTEMML-741])

+1

Nakul Jindal


On Thu, Oct 20, 2016 at 12:18 PM, <du...@gmail.com> wrote:

> Okay I've been testing the release candidate on a large-scale problem, and
> I'm currently running into a "java.lang.NegativeArraySizeException" in
> the SparseBlockMCSR that I do not believe was present previously. I'm
> currently investigating, and will post again soon.
>
> On another note, I successfully ran all of the Python tests on both Python
> 2.7 and 3.5.
>
> -Mike
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
> >
> > Yes - that is correct for test cases involving ID column for
> DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
> MLContextTest. The four failures for MLContextFrameTest were slightly
> different and involve similar fix as done for FrameConverterTest under
> [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to
> incorporate schema information when converting to JavaRDD<Row>.
> >
> > Thanks,
> > Glenn
> >
> > Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues were
> only caused by wrong tests that used an invalid ID schema or populated
> >
> > From: Matthias Boehm <mb...@googlemail.com>
> > To: dev@systemml.incubator.apache.org
> > Date: 10/19/2016 12:36 PM
> > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> >
> >
> >
> >
> > Glenn, all these issues were only caused by wrong tests that used an
> > invalid ID schema or populated this column incorrectly, right? If so,
> > then I think it's fine to release. However, if we touch it anyway, we
> > should globally change the ID schema from double to long, which is more
> > intuitive when created by hand.
> >
> > Regards,
> > Matthias
> >
> > On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> > > OK, so I think it's my understanding that for the 'src' release for
> rc3,
> > > the pom is using Spark 1.4 and the test suite passes for Spark 1.4, so
> this
> > > issue being discussed regarding test cases on Spark 1.6 is not a
> blocker
> > > for this release since the 'src' release builds and all tests pass.
> > >
> > > If this is not correct, could someone please correct me?
> > >
> > > Deron
> > >
> > >
> > > On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <
> luckbr1975@gmail.com>
> > > wrote:
> > >
> > >> if tests are consistently failing, then we should cancel the RC and
> either
> > >> fix the test or mark it as @ignored.
> > >>
> > >> Intermittent fails might be ok, but it's a community decision.
> > >>
> > >> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <
> deroneriksson@gmail.com>
> > >> wrote:
> > >>
> > >>> I believe that for an Apache release, our test suite is supposed to
> pass
> > >>> (although I'm pretty sure random test fails can be ignored).
> > >>>
> > >>> See 2.1 of Release Check List here:
> > >>> http://incubator.apache.org/guides/releasemanagement.html#check-list
> > >>>
> > >>> "2.1 Build is successful including automated tests.
> > >>> The expanded source archive is expected to build and pass tests."
> > >>>
> > >>> Luciano, do you happen to know if some test failures are acceptable
> since
> > >>> our test suite is so enormous (6300+ tests)?
> > >>>
> > >>> Deron
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
> > >>> wrote:
> > >>>
> > >>>> It's a nice-to-have but not a release blocker.
> > >>>>
> > >>>> Thanks,
> > >>>> Glenn
> > >>>>
> > >>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
> 05:38:26
> > >>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
> > >> Niketan
> > >>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> > >>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11
> releas
> > >>>>
> > >>>> From: Niketan Pansare/Almaden/IBM@IBMUS
> > >>>> To: dev@systemml.incubator.apache.org
> > >>>> Date: 10/18/2016 05:38 PM
> > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >>>> ------------------------------
> > >>>>
> > >>>>
> > >>>>
> > >>>> Glenn: Would you prefer to have
> > >>>> *https://github.com/apache/incubator-systemml/pull/269*
> > >>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> > >> release
> > >>> ?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Niketan Pansare
> > >>>> IBM Almaden Research Center
> > >>>> E-mail: npansar At us.ibm.com
> > >>>> *http://researcher.watson.ibm.com/researcher/view.php?
> > >> person=us-npansar*
> > >>>> <http://researcher.watson.ibm.com/researcher/view.php?
> > >> person=us-npansar>
> > >>>>
> > >>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> > >>>> correction on the RC tag name (the actual tag hash is correct):
> > >>>>
> > >>>> From: Luciano Resende <lu...@gmail.com>
> > >>>> To: dev@systemml.incubator.apache.org
> > >>>> Date: 10/17/2016 09:06 PM
> > >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > >>>> ------------------------------
> > >>>>
> > >>>>
> > >>>>
> > >>>> Please note the minor correction on the RC tag name (the actual tag
> > >> hash
> > >>> is
> > >>>> correct):
> > >>>>
> > >>>> The tag to be voted on is v0.11.0-incubating-rc3 (
> > >>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > >>>>
> > >>>>
> > >>>> And off course, my +1
> > >>>>
> > >>>>
> > >>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> > >> luckbr1975@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>>
> > >>>>> Please vote on releasing the following candidate as Apache SystemML
> > >>>>> version 0.11.0-incubating !
> > >>>>>
> > >>>>> The vote is open for at least 72 hours and passes if a majority of
> at
> > >>>>> least 3 +1 PMC votes are cast.
> > >>>>>
> > >>>>> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> > >>>>> [ ] -1 Do not release this package because ...
> > >>>>>
> > >>>>> To learn more about Apache SystemML, please see
> > >>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
> > >>>>>
> > >>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
> > >>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > >>>>>
> > >>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > >>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > >>>>> 0134b3af6d373c254ee084a6d28cc3
> > >>>>>
> > >>>>> The release artifacts can be found at :
> > >>>>>
> > >>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > >>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > >>>>> 11.0-incubating-rc3/
> > >>>>>
> > >>>>> The maven release artifacts, including signatures, digests, etc.
> can
> > >> be
> > >>>>> found at:
> > >>>>>
> > >>>>>
> > >>>> *https://repository.apache.org/content/repositories/
> > >>> orgapachesystemml-1009/*
> > >>>> <https://repository.apache.org/content/repositories/
> > >>> orgapachesystemml-1009/>
> > >>>>>
> > >>>>>
> > >>>>> =====================================
> > >>>>> == Apache Incubator release policy ==
> > >>>>> =====================================
> > >>>>> Please find below the guide to release management during
> incubation:
> > >>>>> *http://incubator.apache.org/guides/releasemanagement.html*
> > >>>> <http://incubator.apache.org/guides/releasemanagement.html>
> > >>>>>
> > >>>>> =======================================
> > >>>>> == How can I help test this release? ==
> > >>>>> =======================================
> > >>>>> If you are a SystemML user, you can help us test this release by
> > >> taking
> > >>>> an
> > >>>>> existing Algorithm or workload and running on this release
> candidate,
> > >>>> then
> > >>>>> reporting any regressions.
> > >>>>>
> > >>>>> ================================================
> > >>>>> == What justifies a -1 vote for this release? ==
> > >>>>> ================================================
> > >>>>> -1 votes should only occur for significant stop-ship bugs or legal
> > >>>>> related issues (e.g. wrong license, missing header files, etc).
> Minor
> > >>>> bugs
> > >>>>> or regressions should not block this release.
> > >>>>>
> > >>>>>
> > >>>>
> > >>>> --
> > >>>> Luciano Resende
> > >>>> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
> > >>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Luciano Resende
> > >> http://twitter.com/lresende1975
> > >> http://lresende.blogspot.com/
> > >>
> > >
> >
> >
> >
> >
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by du...@gmail.com.
Okay I've been testing the release candidate on a large-scale problem, and I'm currently running into a "java.lang.NegativeArraySizeException" in the SparseBlockMCSR that I do not believe was present previously. I'm currently investigating, and will post again soon.

On another note, I successfully ran all of the Python tests on both Python 2.7 and 3.5.

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Oct 19, 2016, at 2:46 PM, Glenn Weidner <gw...@us.ibm.com> wrote:
> 
> Yes - that is correct for test cases involving ID column for DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest, MLContextTest. The four failures for MLContextFrameTest were slightly different and involve similar fix as done for FrameConverterTest under [SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to incorporate schema information when converting to JavaRDD<Row>.
> 
> Thanks,
> Glenn
> 
> Matthias Boehm ---10/19/2016 12:36:04 PM---Glenn, all these issues were only caused by wrong tests that used an invalid ID schema or populated
> 
> From: Matthias Boehm <mb...@googlemail.com>
> To: dev@systemml.incubator.apache.org
> Date: 10/19/2016 12:36 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> 
> 
> 
> 
> Glenn, all these issues were only caused by wrong tests that used an 
> invalid ID schema or populated this column incorrectly, right? If so, 
> then I think it's fine to release. However, if we touch it anyway, we 
> should globally change the ID schema from double to long, which is more 
> intuitive when created by hand.
> 
> Regards,
> Matthias
> 
> On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> > OK, so I think it's my understanding that for the 'src' release for rc3,
> > the pom is using Spark 1.4 and the test suite passes for Spark 1.4, so this
> > issue being discussed regarding test cases on Spark 1.6 is not a blocker
> > for this release since the 'src' release builds and all tests pass.
> >
> > If this is not correct, could someone please correct me?
> >
> > Deron
> >
> >
> > On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <lu...@gmail.com>
> > wrote:
> >
> >> if tests are consistently failing, then we should cancel the RC and either
> >> fix the test or mark it as @ignored.
> >>
> >> Intermittent fails might be ok, but it's a community decision.
> >>
> >> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <de...@gmail.com>
> >> wrote:
> >>
> >>> I believe that for an Apache release, our test suite is supposed to pass
> >>> (although I'm pretty sure random test fails can be ignored).
> >>>
> >>> See 2.1 of Release Check List here:
> >>> http://incubator.apache.org/guides/releasemanagement.html#check-list
> >>>
> >>> "2.1 Build is successful including automated tests.
> >>> The expanded source archive is expected to build and pass tests."
> >>>
> >>> Luciano, do you happen to know if some test failures are acceptable since
> >>> our test suite is so enormous (6300+ tests)?
> >>>
> >>> Deron
> >>>
> >>>
> >>>
> >>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
> >>> wrote:
> >>>
> >>>> It's a nice-to-have but not a release blocker.
> >>>>
> >>>> Thanks,
> >>>> Glenn
> >>>>
> >>>> [image: Inactive hide details for Niketan Pansare---10/18/2016 05:38:26
> >>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
> >> Niketan
> >>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> >>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11 releas
> >>>>
> >>>> From: Niketan Pansare/Almaden/IBM@IBMUS
> >>>> To: dev@systemml.incubator.apache.org
> >>>> Date: 10/18/2016 05:38 PM
> >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> >>>> ------------------------------
> >>>>
> >>>>
> >>>>
> >>>> Glenn: Would you prefer to have
> >>>> *https://github.com/apache/incubator-systemml/pull/269*
> >>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> >> release
> >>> ?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Niketan Pansare
> >>>> IBM Almaden Research Center
> >>>> E-mail: npansar At us.ibm.com
> >>>> *http://researcher.watson.ibm.com/researcher/view.php?
> >> person=us-npansar*
> >>>> <http://researcher.watson.ibm.com/researcher/view.php?
> >> person=us-npansar>
> >>>>
> >>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> >>>> correction on the RC tag name (the actual tag hash is correct):
> >>>>
> >>>> From: Luciano Resende <lu...@gmail.com>
> >>>> To: dev@systemml.incubator.apache.org
> >>>> Date: 10/17/2016 09:06 PM
> >>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> >>>> ------------------------------
> >>>>
> >>>>
> >>>>
> >>>> Please note the minor correction on the RC tag name (the actual tag
> >> hash
> >>> is
> >>>> correct):
> >>>>
> >>>> The tag to be voted on is v0.11.0-incubating-rc3 (
> >>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> >>>>
> >>>>
> >>>> And off course, my +1
> >>>>
> >>>>
> >>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> >> luckbr1975@gmail.com>
> >>>> wrote:
> >>>>
> >>>>>
> >>>>> Please vote on releasing the following candidate as Apache SystemML
> >>>>> version 0.11.0-incubating !
> >>>>>
> >>>>> The vote is open for at least 72 hours and passes if a majority of at
> >>>>> least 3 +1 PMC votes are cast.
> >>>>>
> >>>>> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> >>>>> [ ] -1 Do not release this package because ...
> >>>>>
> >>>>> To learn more about Apache SystemML, please see
> >>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
> >>>>>
> >>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
> >>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
> >>>>>
> >>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> >>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> >>>>> 0134b3af6d373c254ee084a6d28cc3
> >>>>>
> >>>>> The release artifacts can be found at :
> >>>>>
> >>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> >>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> >>>>> 11.0-incubating-rc3/
> >>>>>
> >>>>> The maven release artifacts, including signatures, digests, etc. can
> >> be
> >>>>> found at:
> >>>>>
> >>>>>
> >>>> *https://repository.apache.org/content/repositories/
> >>> orgapachesystemml-1009/*
> >>>> <https://repository.apache.org/content/repositories/
> >>> orgapachesystemml-1009/>
> >>>>>
> >>>>>
> >>>>> =====================================
> >>>>> == Apache Incubator release policy ==
> >>>>> =====================================
> >>>>> Please find below the guide to release management during incubation:
> >>>>> *http://incubator.apache.org/guides/releasemanagement.html*
> >>>> <http://incubator.apache.org/guides/releasemanagement.html>
> >>>>>
> >>>>> =======================================
> >>>>> == How can I help test this release? ==
> >>>>> =======================================
> >>>>> If you are a SystemML user, you can help us test this release by
> >> taking
> >>>> an
> >>>>> existing Algorithm or workload and running on this release candidate,
> >>>> then
> >>>>> reporting any regressions.
> >>>>>
> >>>>> ================================================
> >>>>> == What justifies a -1 vote for this release? ==
> >>>>> ================================================
> >>>>> -1 votes should only occur for significant stop-ship bugs or legal
> >>>>> related issues (e.g. wrong license, missing header files, etc). Minor
> >>>> bugs
> >>>>> or regressions should not block this release.
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> Luciano Resende
> >>>> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
> >>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Luciano Resende
> >> http://twitter.com/lresende1975
> >> http://lresende.blogspot.com/
> >>
> >
> 
> 
> 
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Glenn Weidner <gw...@us.ibm.com>.
Yes - that is correct for test cases involving ID column for
DataFrameVectorFrameConversionTest, DataFrameVectorScriptTest,
MLContextTest.  The four failures for MLContextFrameTest were slightly
different and involve similar fix as done for FrameConverterTest under
[SYSTEMML-568] where FrameRDDConverterUtils.csvToRowRDDused to incorporate
schema information when converting to JavaRDD<Row>.

Thanks,
Glenn



From:	Matthias Boehm <mb...@googlemail.com>
To:	dev@systemml.incubator.apache.org
Date:	10/19/2016 12:36 PM
Subject:	Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Glenn, all these issues were only caused by wrong tests that used an
invalid ID schema or populated this column incorrectly, right? If so,
then I think it's fine to release. However, if we touch it anyway, we
should globally change the ID schema from double to long, which is more
intuitive when created by hand.

Regards,
Matthias

On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> OK, so I think it's my understanding that for the 'src' release for rc3,
> the pom is using Spark 1.4 and the test suite passes for Spark 1.4, so
this
> issue being discussed regarding test cases on Spark 1.6 is not a blocker
> for this release since the 'src' release builds and all tests pass.
>
> If this is not correct, could someone please correct me?
>
> Deron
>
>
> On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <lu...@gmail.com>
> wrote:
>
>> if tests are consistently failing, then we should cancel the RC and
either
>> fix the test or mark it as @ignored.
>>
>> Intermittent fails might be ok, but it's a community decision.
>>
>> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson
<de...@gmail.com>
>> wrote:
>>
>>> I believe that for an Apache release, our test suite is supposed to
pass
>>> (although I'm pretty sure random test fails can be ignored).
>>>
>>> See 2.1 of Release Check List here:
>>> http://incubator.apache.org/guides/releasemanagement.html#check-list
>>>
>>> "2.1 Build is successful including automated tests.
>>> The expanded source archive is expected to build and pass tests."
>>>
>>> Luciano, do you happen to know if some test failures are acceptable
since
>>> our test suite is so enormous (6300+ tests)?
>>>
>>> Deron
>>>
>>>
>>>
>>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
>>> wrote:
>>>
>>>> It's a nice-to-have but not a release blocker.
>>>>
>>>> Thanks,
>>>> Glenn
>>>>
>>>> [image: Inactive hide details for Niketan Pansare---10/18/2016
05:38:26
>>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
>> Niketan
>>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
>>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11 releas
>>>>
>>>> From: Niketan Pansare/Almaden/IBM@IBMUS
>>>> To: dev@systemml.incubator.apache.org
>>>> Date: 10/18/2016 05:38 PM
>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>> Glenn: Would you prefer to have
>>>> *https://github.com/apache/incubator-systemml/pull/269*
>>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
>> release
>>> ?
>>>>
>>>> Thanks,
>>>>
>>>> Niketan Pansare
>>>> IBM Almaden Research Center
>>>> E-mail: npansar At us.ibm.com
>>>> *http://researcher.watson.ibm.com/researcher/view.php?
>> person=us-npansar*
>>>> <http://researcher.watson.ibm.com/researcher/view.php?
>> person=us-npansar>
>>>>
>>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
>>>> correction on the RC tag name (the actual tag hash is correct):
>>>>
>>>> From: Luciano Resende <lu...@gmail.com>
>>>> To: dev@systemml.incubator.apache.org
>>>> Date: 10/17/2016 09:06 PM
>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>> Please note the minor correction on the RC tag name (the actual tag
>> hash
>>> is
>>>> correct):
>>>>
>>>> The tag to be voted on is v0.11.0-incubating-rc3 (
>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>
>>>>
>>>> And off course, my +1
>>>>
>>>>
>>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
>> luckbr1975@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Please vote on releasing the following candidate as Apache SystemML
>>>>> version 0.11.0-incubating !
>>>>>
>>>>> The vote is open for at least 72 hours and passes if a majority of at
>>>>> least 3 +1 PMC votes are cast.
>>>>>
>>>>> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
>>>>> [ ] -1 Do not release this package because ...
>>>>>
>>>>> To learn more about Apache SystemML, please see
>>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
>>>>>
>>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
>>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>>
>>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
>>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
>>>>> 0134b3af6d373c254ee084a6d28cc3
>>>>>
>>>>> The release artifacts can be found at :
>>>>>
>>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
>>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
>>>>> 11.0-incubating-rc3/
>>>>>
>>>>> The maven release artifacts, including signatures, digests, etc. can
>> be
>>>>> found at:
>>>>>
>>>>>
>>>> *https://repository.apache.org/content/repositories/
>>> orgapachesystemml-1009/*
>>>> <https://repository.apache.org/content/repositories/
>>> orgapachesystemml-1009/>
>>>>>
>>>>>
>>>>> =====================================
>>>>> == Apache Incubator release policy ==
>>>>> =====================================
>>>>> Please find below the guide to release management during incubation:
>>>>> *http://incubator.apache.org/guides/releasemanagement.html*
>>>> <http://incubator.apache.org/guides/releasemanagement.html>
>>>>>
>>>>> =======================================
>>>>> == How can I help test this release? ==
>>>>> =======================================
>>>>> If you are a SystemML user, you can help us test this release by
>> taking
>>>> an
>>>>> existing Algorithm or workload and running on this release candidate,
>>>> then
>>>>> reporting any regressions.
>>>>>
>>>>> ================================================
>>>>> == What justifies a -1 vote for this release? ==
>>>>> ================================================
>>>>> -1 votes should only occur for significant stop-ship bugs or legal
>>>>> related issues (e.g. wrong license, missing header files, etc). Minor
>>>> bugs
>>>>> or regressions should not block this release.
>>>>>
>>>>>
>>>>
>>>> --
>>>> Luciano Resende
>>>> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
>>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Matthias Boehm <mb...@googlemail.com>.
Glenn, all these issues were only caused by wrong tests that used an 
invalid ID schema or populated this column incorrectly, right? If so, 
then I think it's fine to release. However, if we touch it anyway, we 
should globally change the ID schema from double to long, which is more 
intuitive when created by hand.

Regards,
Matthias

On 10/19/2016 8:30 PM, Deron Eriksson wrote:
> OK, so I think it's my understanding that for the 'src' release for rc3,
> the pom is using Spark 1.4 and the test suite passes for Spark 1.4, so this
> issue being discussed regarding test cases on Spark 1.6 is not a blocker
> for this release since the 'src' release builds and all tests pass.
>
> If this is not correct, could someone please correct me?
>
> Deron
>
>
> On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <lu...@gmail.com>
> wrote:
>
>> if tests are consistently failing, then we should cancel the RC and either
>> fix the test or mark it as @ignored.
>>
>> Intermittent fails might be ok, but it's a community decision.
>>
>> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <de...@gmail.com>
>> wrote:
>>
>>> I believe that for an Apache release, our test suite is supposed to pass
>>> (although I'm pretty sure random test fails can be ignored).
>>>
>>> See 2.1 of Release Check List here:
>>> http://incubator.apache.org/guides/releasemanagement.html#check-list
>>>
>>> "2.1 Build is successful including automated tests.
>>> The expanded source archive is expected to build and pass tests."
>>>
>>> Luciano, do you happen to know if some test failures are acceptable since
>>> our test suite is so enormous (6300+ tests)?
>>>
>>> Deron
>>>
>>>
>>>
>>> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
>>> wrote:
>>>
>>>> It's a nice-to-have but not a release blocker.
>>>>
>>>> Thanks,
>>>> Glenn
>>>>
>>>> [image: Inactive hide details for Niketan Pansare---10/18/2016 05:38:26
>>>> PM---Glenn: Would you prefer to have https://github.com/apache/]
>> Niketan
>>>> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
>>>> https://github.com/apache/incubator-systemml/pull/269 in 0.11 releas
>>>>
>>>> From: Niketan Pansare/Almaden/IBM@IBMUS
>>>> To: dev@systemml.incubator.apache.org
>>>> Date: 10/18/2016 05:38 PM
>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>> Glenn: Would you prefer to have
>>>> *https://github.com/apache/incubator-systemml/pull/269*
>>>> <https://github.com/apache/incubator-systemml/pull/269> in 0.11
>> release
>>> ?
>>>>
>>>> Thanks,
>>>>
>>>> Niketan Pansare
>>>> IBM Almaden Research Center
>>>> E-mail: npansar At us.ibm.com
>>>> *http://researcher.watson.ibm.com/researcher/view.php?
>> person=us-npansar*
>>>> <http://researcher.watson.ibm.com/researcher/view.php?
>> person=us-npansar>
>>>>
>>>> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
>>>> correction on the RC tag name (the actual tag hash is correct):
>>>>
>>>> From: Luciano Resende <lu...@gmail.com>
>>>> To: dev@systemml.incubator.apache.org
>>>> Date: 10/17/2016 09:06 PM
>>>> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>> Please note the minor correction on the RC tag name (the actual tag
>> hash
>>> is
>>>> correct):
>>>>
>>>> The tag to be voted on is v0.11.0-incubating-rc3 (
>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>
>>>>
>>>> And off course, my +1
>>>>
>>>>
>>>> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
>> luckbr1975@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Please vote on releasing the following candidate as Apache SystemML
>>>>> version 0.11.0-incubating !
>>>>>
>>>>> The vote is open for at least 72 hours and passes if a majority of at
>>>>> least 3 +1 PMC votes are cast.
>>>>>
>>>>> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
>>>>> [ ] -1 Do not release this package because ...
>>>>>
>>>>> To learn more about Apache SystemML, please see
>>>>> *http://systemml.apache.org/* <http://systemml.apache.org/>
>>>>>
>>>>> The tag to be voted on is v0.11.0-incubating-rc1 (
>>>>> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>>>>>
>>>>> *https://github.com/apache/incubator-systemml/tree/1baebfde40*
>>>> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
>>>>> 0134b3af6d373c254ee084a6d28cc3
>>>>>
>>>>> The release artifacts can be found at :
>>>>>
>>>>> *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
>>>> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
>>>>> 11.0-incubating-rc3/
>>>>>
>>>>> The maven release artifacts, including signatures, digests, etc. can
>> be
>>>>> found at:
>>>>>
>>>>>
>>>> *https://repository.apache.org/content/repositories/
>>> orgapachesystemml-1009/*
>>>> <https://repository.apache.org/content/repositories/
>>> orgapachesystemml-1009/>
>>>>>
>>>>>
>>>>> =====================================
>>>>> == Apache Incubator release policy ==
>>>>> =====================================
>>>>> Please find below the guide to release management during incubation:
>>>>> *http://incubator.apache.org/guides/releasemanagement.html*
>>>> <http://incubator.apache.org/guides/releasemanagement.html>
>>>>>
>>>>> =======================================
>>>>> == How can I help test this release? ==
>>>>> =======================================
>>>>> If you are a SystemML user, you can help us test this release by
>> taking
>>>> an
>>>>> existing Algorithm or workload and running on this release candidate,
>>>> then
>>>>> reporting any regressions.
>>>>>
>>>>> ================================================
>>>>> == What justifies a -1 vote for this release? ==
>>>>> ================================================
>>>>> -1 votes should only occur for significant stop-ship bugs or legal
>>>>> related issues (e.g. wrong license, missing header files, etc). Minor
>>>> bugs
>>>>> or regressions should not block this release.
>>>>>
>>>>>
>>>>
>>>> --
>>>> Luciano Resende
>>>> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
>>>> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Deron Eriksson <de...@gmail.com>.
OK, so I think it's my understanding that for the 'src' release for rc3,
the pom is using Spark 1.4 and the test suite passes for Spark 1.4, so this
issue being discussed regarding test cases on Spark 1.6 is not a blocker
for this release since the 'src' release builds and all tests pass.

If this is not correct, could someone please correct me?

Deron


On Wed, Oct 19, 2016 at 11:17 AM, Luciano Resende <lu...@gmail.com>
wrote:

> if tests are consistently failing, then we should cancel the RC and either
> fix the test or mark it as @ignored.
>
> Intermittent fails might be ok, but it's a community decision.
>
> On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <de...@gmail.com>
> wrote:
>
> > I believe that for an Apache release, our test suite is supposed to pass
> > (although I'm pretty sure random test fails can be ignored).
> >
> > See 2.1 of Release Check List here:
> > http://incubator.apache.org/guides/releasemanagement.html#check-list
> >
> > "2.1 Build is successful including automated tests.
> > The expanded source archive is expected to build and pass tests."
> >
> > Luciano, do you happen to know if some test failures are acceptable since
> > our test suite is so enormous (6300+ tests)?
> >
> > Deron
> >
> >
> >
> > On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
> > wrote:
> >
> > > It's a nice-to-have but not a release blocker.
> > >
> > > Thanks,
> > > Glenn
> > >
> > > [image: Inactive hide details for Niketan Pansare---10/18/2016 05:38:26
> > > PM---Glenn: Would you prefer to have https://github.com/apache/]
> Niketan
> > > Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> > > https://github.com/apache/incubator-systemml/pull/269 in 0.11 releas
> > >
> > > From: Niketan Pansare/Almaden/IBM@IBMUS
> > > To: dev@systemml.incubator.apache.org
> > > Date: 10/18/2016 05:38 PM
> > > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > ------------------------------
> > >
> > >
> > >
> > > Glenn: Would you prefer to have
> > > *https://github.com/apache/incubator-systemml/pull/269*
> > > <https://github.com/apache/incubator-systemml/pull/269> in 0.11
> release
> > ?
> > >
> > > Thanks,
> > >
> > > Niketan Pansare
> > > IBM Almaden Research Center
> > > E-mail: npansar At us.ibm.com
> > > *http://researcher.watson.ibm.com/researcher/view.php?
> person=us-npansar*
> > > <http://researcher.watson.ibm.com/researcher/view.php?
> person=us-npansar>
> > >
> > > Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> > > correction on the RC tag name (the actual tag hash is correct):
> > >
> > > From: Luciano Resende <lu...@gmail.com>
> > > To: dev@systemml.incubator.apache.org
> > > Date: 10/17/2016 09:06 PM
> > > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > > ------------------------------
> > >
> > >
> > >
> > > Please note the minor correction on the RC tag name (the actual tag
> hash
> > is
> > > correct):
> > >
> > > The tag to be voted on is v0.11.0-incubating-rc3 (
> > > 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > >
> > >
> > > And off course, my +1
> > >
> > >
> > > On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <
> luckbr1975@gmail.com>
> > > wrote:
> > >
> > > >
> > > > Please vote on releasing the following candidate as Apache SystemML
> > > > version 0.11.0-incubating !
> > > >
> > > > The vote is open for at least 72 hours and passes if a majority of at
> > > > least 3 +1 PMC votes are cast.
> > > >
> > > > [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> > > > [ ] -1 Do not release this package because ...
> > > >
> > > > To learn more about Apache SystemML, please see
> > > > *http://systemml.apache.org/* <http://systemml.apache.org/>
> > > >
> > > > The tag to be voted on is v0.11.0-incubating-rc1 (
> > > > 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > > >
> > > > *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > > <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > > > 0134b3af6d373c254ee084a6d28cc3
> > > >
> > > > The release artifacts can be found at :
> > > >
> > > > *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > > <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > > > 11.0-incubating-rc3/
> > > >
> > > > The maven release artifacts, including signatures, digests, etc. can
> be
> > > > found at:
> > > >
> > > >
> > > *https://repository.apache.org/content/repositories/
> > orgapachesystemml-1009/*
> > > <https://repository.apache.org/content/repositories/
> > orgapachesystemml-1009/>
> > > >
> > > >
> > > > =====================================
> > > > == Apache Incubator release policy ==
> > > > =====================================
> > > > Please find below the guide to release management during incubation:
> > > > *http://incubator.apache.org/guides/releasemanagement.html*
> > > <http://incubator.apache.org/guides/releasemanagement.html>
> > > >
> > > > =======================================
> > > > == How can I help test this release? ==
> > > > =======================================
> > > > If you are a SystemML user, you can help us test this release by
> taking
> > > an
> > > > existing Algorithm or workload and running on this release candidate,
> > > then
> > > > reporting any regressions.
> > > >
> > > > ================================================
> > > > == What justifies a -1 vote for this release? ==
> > > > ================================================
> > > > -1 votes should only occur for significant stop-ship bugs or legal
> > > > related issues (e.g. wrong license, missing header files, etc). Minor
> > > bugs
> > > > or regressions should not block this release.
> > > >
> > > >
> > >
> > > --
> > > Luciano Resende
> > > *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
> > > *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Luciano Resende <lu...@gmail.com>.
if tests are consistently failing, then we should cancel the RC and either
fix the test or mark it as @ignored.

Intermittent fails might be ok, but it's a community decision.

On Wed, Oct 19, 2016 at 10:50 AM, Deron Eriksson <de...@gmail.com>
wrote:

> I believe that for an Apache release, our test suite is supposed to pass
> (although I'm pretty sure random test fails can be ignored).
>
> See 2.1 of Release Check List here:
> http://incubator.apache.org/guides/releasemanagement.html#check-list
>
> "2.1 Build is successful including automated tests.
> The expanded source archive is expected to build and pass tests."
>
> Luciano, do you happen to know if some test failures are acceptable since
> our test suite is so enormous (6300+ tests)?
>
> Deron
>
>
>
> On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com>
> wrote:
>
> > It's a nice-to-have but not a release blocker.
> >
> > Thanks,
> > Glenn
> >
> > [image: Inactive hide details for Niketan Pansare---10/18/2016 05:38:26
> > PM---Glenn: Would you prefer to have https://github.com/apache/]Niketan
> > Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> > https://github.com/apache/incubator-systemml/pull/269 in 0.11 releas
> >
> > From: Niketan Pansare/Almaden/IBM@IBMUS
> > To: dev@systemml.incubator.apache.org
> > Date: 10/18/2016 05:38 PM
> > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > ------------------------------
> >
> >
> >
> > Glenn: Would you prefer to have
> > *https://github.com/apache/incubator-systemml/pull/269*
> > <https://github.com/apache/incubator-systemml/pull/269> in 0.11 release
> ?
> >
> > Thanks,
> >
> > Niketan Pansare
> > IBM Almaden Research Center
> > E-mail: npansar At us.ibm.com
> > *http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar*
> > <http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar>
> >
> > Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> > correction on the RC tag name (the actual tag hash is correct):
> >
> > From: Luciano Resende <lu...@gmail.com>
> > To: dev@systemml.incubator.apache.org
> > Date: 10/17/2016 09:06 PM
> > Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> > ------------------------------
> >
> >
> >
> > Please note the minor correction on the RC tag name (the actual tag hash
> is
> > correct):
> >
> > The tag to be voted on is v0.11.0-incubating-rc3 (
> > 1baebfde400134b3af6d373c254ee084a6d28cc3)
> >
> >
> > And off course, my +1
> >
> >
> > On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <lu...@gmail.com>
> > wrote:
> >
> > >
> > > Please vote on releasing the following candidate as Apache SystemML
> > > version 0.11.0-incubating !
> > >
> > > The vote is open for at least 72 hours and passes if a majority of at
> > > least 3 +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> > > [ ] -1 Do not release this package because ...
> > >
> > > To learn more about Apache SystemML, please see
> > > *http://systemml.apache.org/* <http://systemml.apache.org/>
> > >
> > > The tag to be voted on is v0.11.0-incubating-rc1 (
> > > 1baebfde400134b3af6d373c254ee084a6d28cc3)
> > >
> > > *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> > <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > > 0134b3af6d373c254ee084a6d28cc3
> > >
> > > The release artifacts can be found at :
> > >
> > > *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> > <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > > 11.0-incubating-rc3/
> > >
> > > The maven release artifacts, including signatures, digests, etc. can be
> > > found at:
> > >
> > >
> > *https://repository.apache.org/content/repositories/
> orgapachesystemml-1009/*
> > <https://repository.apache.org/content/repositories/
> orgapachesystemml-1009/>
> > >
> > >
> > > =====================================
> > > == Apache Incubator release policy ==
> > > =====================================
> > > Please find below the guide to release management during incubation:
> > > *http://incubator.apache.org/guides/releasemanagement.html*
> > <http://incubator.apache.org/guides/releasemanagement.html>
> > >
> > > =======================================
> > > == How can I help test this release? ==
> > > =======================================
> > > If you are a SystemML user, you can help us test this release by taking
> > an
> > > existing Algorithm or workload and running on this release candidate,
> > then
> > > reporting any regressions.
> > >
> > > ================================================
> > > == What justifies a -1 vote for this release? ==
> > > ================================================
> > > -1 votes should only occur for significant stop-ship bugs or legal
> > > related issues (e.g. wrong license, missing header files, etc). Minor
> > bugs
> > > or regressions should not block this release.
> > >
> > >
> >
> > --
> > Luciano Resende
> > *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
> > *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
> >
> >
> >
> >
> >
> >
>



-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Deron Eriksson <de...@gmail.com>.
I believe that for an Apache release, our test suite is supposed to pass
(although I'm pretty sure random test fails can be ignored).

See 2.1 of Release Check List here:
http://incubator.apache.org/guides/releasemanagement.html#check-list

"2.1 Build is successful including automated tests.
The expanded source archive is expected to build and pass tests."

Luciano, do you happen to know if some test failures are acceptable since
our test suite is so enormous (6300+ tests)?

Deron



On Wed, Oct 19, 2016 at 3:24 AM, Glenn Weidner <gw...@us.ibm.com> wrote:

> It's a nice-to-have but not a release blocker.
>
> Thanks,
> Glenn
>
> [image: Inactive hide details for Niketan Pansare---10/18/2016 05:38:26
> PM---Glenn: Would you prefer to have https://github.com/apache/]Niketan
> Pansare---10/18/2016 05:38:26 PM---Glenn: Would you prefer to have
> https://github.com/apache/incubator-systemml/pull/269 in 0.11 releas
>
> From: Niketan Pansare/Almaden/IBM@IBMUS
> To: dev@systemml.incubator.apache.org
> Date: 10/18/2016 05:38 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> ------------------------------
>
>
>
> Glenn: Would you prefer to have
> *https://github.com/apache/incubator-systemml/pull/269*
> <https://github.com/apache/incubator-systemml/pull/269> in 0.11 release ?
>
> Thanks,
>
> Niketan Pansare
> IBM Almaden Research Center
> E-mail: npansar At us.ibm.com
> *http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar*
> <http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar>
>
> Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
> correction on the RC tag name (the actual tag hash is correct):
>
> From: Luciano Resende <lu...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 10/17/2016 09:06 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)
> ------------------------------
>
>
>
> Please note the minor correction on the RC tag name (the actual tag hash is
> correct):
>
> The tag to be voted on is v0.11.0-incubating-rc3 (
> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>
>
> And off course, my +1
>
>
> On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <lu...@gmail.com>
> wrote:
>
> >
> > Please vote on releasing the following candidate as Apache SystemML
> > version 0.11.0-incubating !
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> > least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache SystemML, please see
> > *http://systemml.apache.org/* <http://systemml.apache.org/>
> >
> > The tag to be voted on is v0.11.0-incubating-rc1 (
> > 1baebfde400134b3af6d373c254ee084a6d28cc3)
> >
> > *https://github.com/apache/incubator-systemml/tree/1baebfde40*
> <https://github.com/apache/incubator-systemml/tree/1baebfde40>
> > 0134b3af6d373c254ee084a6d28cc3
> >
> > The release artifacts can be found at :
> >
> > *https://dist.apache.org/repos/dist/dev/incubator/systemml/0*
> <https://dist.apache.org/repos/dist/dev/incubator/systemml/0>.
> > 11.0-incubating-rc3/
> >
> > The maven release artifacts, including signatures, digests, etc. can be
> > found at:
> >
> >
> *https://repository.apache.org/content/repositories/orgapachesystemml-1009/*
> <https://repository.apache.org/content/repositories/orgapachesystemml-1009/>
> >
> >
> > =====================================
> > == Apache Incubator release policy ==
> > =====================================
> > Please find below the guide to release management during incubation:
> > *http://incubator.apache.org/guides/releasemanagement.html*
> <http://incubator.apache.org/guides/releasemanagement.html>
> >
> > =======================================
> > == How can I help test this release? ==
> > =======================================
> > If you are a SystemML user, you can help us test this release by taking
> an
> > existing Algorithm or workload and running on this release candidate,
> then
> > reporting any regressions.
> >
> > ================================================
> > == What justifies a -1 vote for this release? ==
> > ================================================
> > -1 votes should only occur for significant stop-ship bugs or legal
> > related issues (e.g. wrong license, missing header files, etc). Minor
> bugs
> > or regressions should not block this release.
> >
> >
>
> --
> Luciano Resende
> *http://twitter.com/lresende1975* <http://twitter.com/lresende1975>
> *http://lresende.blogspot.com/* <http://lresende.blogspot.com/>
>
>
>
>
>
>

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Glenn Weidner <gw...@us.ibm.com>.
It's a nice-to-have but not a release blocker.

Thanks,
Glenn



From:	Niketan Pansare/Almaden/IBM@IBMUS
To:	dev@systemml.incubator.apache.org
Date:	10/18/2016 05:38 PM
Subject:	Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Glenn: Would you prefer to have
https://github.com/apache/incubator-systemml/pull/269 in 0.11 release ?

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar

Luciano Resende ---10/17/2016 09:06:30 PM---Please note the minor
correction on the RC tag name (the actual tag hash is correct):

From: Luciano Resende <lu...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 10/17/2016 09:06 PM
Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Please note the minor correction on the RC tag name (the actual tag hash is
correct):

The tag to be voted on is v0.11.0-incubating-rc3 (
1baebfde400134b3af6d373c254ee084a6d28cc3)


And off course, my +1


On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <lu...@gmail.com>
wrote:

>
> Please vote on releasing the following candidate as Apache SystemML
> version 0.11.0-incubating !
>
> The vote is open for at least 72 hours and passes if a majority of at
> least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache SystemML, please see
> http://systemml.apache.org/
>
> The tag to be voted on is v0.11.0-incubating-rc1 (
> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>
> https://github.com/apache/incubator-systemml/tree/1baebfde40
> 0134b3af6d373c254ee084a6d28cc3
>
> The release artifacts can be found at :
>
> https://dist.apache.org/repos/dist/dev/incubator/systemml/0.
> 11.0-incubating-rc3/
>
> The maven release artifacts, including signatures, digests, etc. can be
> found at:
>
>
https://repository.apache.org/content/repositories/orgapachesystemml-1009/
>
>
> =====================================
> == Apache Incubator release policy ==
> =====================================
> Please find below the guide to release management during incubation:
> http://incubator.apache.org/guides/releasemanagement.html
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a SystemML user, you can help us test this release by taking
an
> existing Algorithm or workload and running on this release candidate,
then
> reporting any regressions.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> -1 votes should only occur for significant stop-ship bugs or legal
> related issues (e.g. wrong license, missing header files, etc). Minor
bugs
> or regressions should not block this release.
>
>

--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Niketan Pansare <np...@us.ibm.com>.
Glenn: Would you prefer to have
https://github.com/apache/incubator-systemml/pull/269 in 0.11 release ?

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar



From:	Luciano Resende <lu...@gmail.com>
To:	dev@systemml.incubator.apache.org
Date:	10/17/2016 09:06 PM
Subject:	Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)



Please note the minor correction on the RC tag name (the actual tag hash is
correct):

The tag to be voted on is v0.11.0-incubating-rc3 (
1baebfde400134b3af6d373c254ee084a6d28cc3)


And off course, my +1


On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <lu...@gmail.com>
wrote:

>
> Please vote on releasing the following candidate as Apache SystemML
> version 0.11.0-incubating !
>
> The vote is open for at least 72 hours and passes if a majority of at
> least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache SystemML, please see
> http://systemml.apache.org/
>
> The tag to be voted on is v0.11.0-incubating-rc1 (
> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>
> https://github.com/apache/incubator-systemml/tree/1baebfde40
> 0134b3af6d373c254ee084a6d28cc3
>
> The release artifacts can be found at :
>
> https://dist.apache.org/repos/dist/dev/incubator/systemml/0.
> 11.0-incubating-rc3/
>
> The maven release artifacts, including signatures, digests, etc. can be
> found at:
>
>
https://repository.apache.org/content/repositories/orgapachesystemml-1009/
>
>
> =====================================
> == Apache Incubator release policy ==
> =====================================
> Please find below the guide to release management during incubation:
> http://incubator.apache.org/guides/releasemanagement.html
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a SystemML user, you can help us test this release by taking
an
> existing Algorithm or workload and running on this release candidate,
then
> reporting any regressions.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> -1 votes should only occur for significant stop-ship bugs or legal
> related issues (e.g. wrong license, missing header files, etc). Minor
bugs
> or regressions should not block this release.
>
>

--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC3)

Posted by Luciano Resende <lu...@gmail.com>.
Please note the minor correction on the RC tag name (the actual tag hash is
correct):

The tag to be voted on is v0.11.0-incubating-rc3 (
1baebfde400134b3af6d373c254ee084a6d28cc3)


And off course, my +1


On Sat, Oct 15, 2016 at 12:27 PM, Luciano Resende <lu...@gmail.com>
wrote:

>
> Please vote on releasing the following candidate as Apache SystemML
> version 0.11.0-incubating !
>
> The vote is open for at least 72 hours and passes if a majority of at
> least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache SystemML 0.11.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache SystemML, please see
> http://systemml.apache.org/
>
> The tag to be voted on is v0.11.0-incubating-rc1 (
> 1baebfde400134b3af6d373c254ee084a6d28cc3)
>
> https://github.com/apache/incubator-systemml/tree/1baebfde40
> 0134b3af6d373c254ee084a6d28cc3
>
> The release artifacts can be found at :
>
> https://dist.apache.org/repos/dist/dev/incubator/systemml/0.
> 11.0-incubating-rc3/
>
> The maven release artifacts, including signatures, digests, etc. can be
> found at:
>
> https://repository.apache.org/content/repositories/orgapachesystemml-1009/
>
>
> =====================================
> == Apache Incubator release policy ==
> =====================================
> Please find below the guide to release management during incubation:
> http://incubator.apache.org/guides/releasemanagement.html
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a SystemML user, you can help us test this release by taking an
> existing Algorithm or workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> -1 votes should only occur for significant stop-ship bugs or legal
> related issues (e.g. wrong license, missing header files, etc). Minor bugs
> or regressions should not block this release.
>
>

-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/