You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Gary Gregory <gg...@apache.org> on 2020/01/18 17:52:13 UTC
[VOTE] Release Apache Commons CSV 1.8 based on RC1
We have fixed quite a few bugs and added some significant enhancements
since Apache Commons CSV 1.7 was released, so I would like to release
Apache Commons CSV 1.8.
Apache Commons CSV 1.8 RC1 is available for review here:
https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
revision 37670)
The Git tag commons-csv-1.8-RC1 commit for this RC is
c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
You may checkout this tag using:
git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
commons-csv-1.8-RC1 commons-csv-1.8-RC1
Maven artifacts are here:
https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
These are the artifacts and their hashes:
#Release SHA-512s
#Sat Jan 18 12:01:01 EST 2020
commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
japicmp clean package site deploy' using:
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk1.8.0_241\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Additional tests with 'mvn -V clean test' using:
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk1.8.0_241\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk-11.0.6
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk-12.0.2
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\jdk-13.0.2
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\openjdk\jdk-14
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
JaCoCo fails on Java 15-EA because it does not know about class file major
version 59:
Caused by: java.lang.IllegalArgumentException: Unsupported class file major
version 59
at
org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Java\apache-maven-3.6.3\bin\..
Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
Files\Java\openjdk\jdk-15
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Details of changes since 1.7 are in the release notes:
https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
Site:
https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
(note some *relative* links are broken and the 1.8 directories are not
yet created - these will be OK once the site is deployed.)
JApiCmp Report (compared to 1.7):
https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
RAT Report:
https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
KEYS:
https://www.apache.org/dist/commons/KEYS
Please review the release candidate and vote.
This vote will close no sooner that 72 hours from now.
[ ] +1 Release these artifacts
[ ] +0 OK, but...
[ ] -0 OK, but really should fix...
[ ] -1 I oppose this release because...
Thank you,
Gary Gregory,
Release Manager (using key 86fdc7e2a11262cb)
For following is intended as a helper and refresher for reviewers.
Validating a release candidate
==============================
These guidelines are NOT complete.
Requirements: Git, Java, Maven.
You can validate a release from a release candidate (RC) tag as follows.
1) Clone and checkout the RC tag
git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
commons-csv-1.8-RC1 commons-csv-1.8-RC1
cd commons-csv-1.8-RC1
2) Check Apache licenses
This step is not required if the site includes a RAT report page which you
then must check.
mvn apache-rat:check
3) Check binary compatibility
Older components still use Apache Clirr:
This step is not required if the site includes a Clirr report page which
you then must check.
mvn clirr:check
Newer components use JApiCmp with the japicmp Maven Profile:
This step is not required if the site includes a JApiCmp report page which
you then must check.
mvn install -DskipTests -P japicmp japicmp:cmp
4) Build the package
mvn -V clean package
You can record the Maven and Java version produced by -V in your VOTE reply.
To gather OS information from a command line:
Windows: ver
Linux: uname -a
5) Build the site for a single module project
Note: Some plugins require the components to be installed instead of
packaged.
mvn site
Check the site reports in:
- Windows: target\site\index.html
- Linux: target/site/index.html
6) Build the site for a multi-module project
mvn site
mvn site:stage
Check the site reports in:
- Windows: target\site\index.html
- Linux: target/site/index.html
-the end-
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
> On 21 Jan 2020, at 00:00, Gary Gregory <ga...@gmail.com> wrote:
>
> On Mon, Jan 20, 2020 at 6:43 PM Rob Tompkins <chtompki@gmail.com <ma...@gmail.com>> wrote:
>
>>
>>
>>> On Jan 20, 2020, at 6:41 PM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>>
>>> On Mon, Jan 20, 2020 at 6:28 PM Gary Gregory <ga...@gmail.com>
>> wrote:
>>>
>>>>> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <alex.d.herbert@gmail.com
>>>
>>>>> wrote:
>>>>>
>>>>> Hi Gary,
>>>>>
>>>>> I raised a few niggles a while back with CSV and the discussion did not
>>>>> receive a response on how to proceed.
>>>>>
>>>>> There is the major bug CSV-248 where the CSVRecord is not Serializable
>>>>> [1]. This requires a decision on what to do to fix it. This bug is
>> still
>>>>> present in 1.8 RC1 as found by FindBugs [2].
>>>>>
>>>>> From what I can see the CSVRecord maintains a reference to the
>> CSVParser.
>>>>> This chain of objects maintained in memory is not serializable and
>> leads
>>>>> back to the original input Reader.
>>>>>
>>>>> I can see from the JApiCmp report that the serial version id was
>> changed
>>>>> for CSVRecord this release so there is still an intention to support
>>>>> serialization. So this should be a blocker.
>>>>>
>>>>> I could not find a serialisation test in the unit tests for CSVRecord.
>>>>> This quick test added to CSVRecordTest fails:
>>>>>
>>>>>
>>>>> @Test
>>>>> public void testSerialization() throws IOException {
>>>>> CSVRecord shortRec;
>>>>> try (final CSVParser parser = CSVParser.parse("a,b",
>>>>> CSVFormat.newFormat(','))) {
>>>>> shortRec = parser.iterator().next();
>>>>> }
>>>>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>>>>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
>>>>> oos.writeObject(shortRec);
>>>>> }
>>>>> }
>>>>>
>>>>> mvn test -Dtest=CSVRecordTest
>>>>>
>>>>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
>>>>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
>>>>> at
>>>>>
>> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>>>>>
>>>>> If I mark the field csvParser as transient it passes. So this is a
>>>>> problem as raised by FindBugs.
>>>>>
>>>>
>>>> Making the field transient would indeed fix this test but... some of the
>>>> record APIs would then fail with NPEs... so we would be kicking the can
>>>> down the road basically. Making the parser serializable is going in the
>>>> wrong direction feature-wise IMO, so let's not go there. A serialization
>>>> proxy would be less worse but should we provide such a thing? I would
>> say
>>>> no. I am OK with making the field transient despite the can kicking, so
>>>> let's do that.
>>>>
>>>
>>> I suppose we should bump the serialVersionUID...
>>
>> +1 to that
>>
>
> The pickle if we do that is that we will be binary incompatible as JApiCmp
> notes:
> MODIFIED (Serializable incompatible(!): serialVersionUID modified) final
> public class org.apache.commons.csv.CSVRecord
If the field is transient then no need to change the serial version ID. That changes when the classes serialised from an old version cannot be deserialised with the new version. Since the ‘new’ parser field is transient it should be ignored.
No one will have a class serialised using version 1.7. It would have thrown an exception as the parser is not serialisable. So we go back to 1.6 and serialise a CSVRecord and check it can be deserialised by 1.8.
Alex
>
> Gary
>
>>
>>
>>> Gary
>>>
>>>
>>>>
>>>> Gary
>>>>
>>>>
>>>>>
>>>>>
>>>>> I also raised [3] the strange implementation of the CSVParser
>>>>> getHeaderNames() which ignores null headers as they cannot be used as
>> a key
>>>>> into the map. However the list of column names could contain the null
>>>>> values. This test currently fails:
>>>>>
>>>>> @Test
>>>>> public void testHeaderNamesWithNull() throws IOException {
>>>>> final Reader in = new
>>>>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
>>>>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>>>>>
>>>>> .withNullString("null")
>>>>>
>>>>> .withAllowMissingColumnNames()
>>>>>
>>>>> .parse(in).iterator();
>>>>> final CSVRecord record = records.next();
>>>>> assertEquals(Arrays.asList("header1", null, "header3"),
>>>>> record.getParser().getHeaderNames());
>>>>> }
>>>>>
>>>>> I am not saying it should pass but at least the documentation should
>>>>> state the behaviour in this edge case. That is the list of header
>> names may
>>>>> be shorter than the number of columns when the parser is configured to
>>>>> allow null headers. I’ve not raised a bug ticket for this as it is
>> open to
>>>>> opinion if this is by design or actually a bug. This issue is still
>> present
>>>>> in 1.8 RC1.
>>>>>
>>>>> Previously I suggested documentation changes for this and another edge
>>>>> case using the header map to be added to the javadoc for
>> getHeaderNames()
>>>>> and getHeaderMap():
>>>>>
>>>>> - Documentation:
>>>>>
>>>>> The mapping is only guaranteed to be a one-to-one mapping if the record
>>>>> was created with a format that does not allow duplicate or null header
>>>>> names. Null headers are excluded from the map and duplicates can only
>> map
>>>>> to 1 column.
>>>>>
>>>>>
>>>>> - Bug / Documentation
>>>>>
>>>>> The CSVParser only stores headers names in a list of header names if
>> they
>>>>> are not null. So the list can be shorter than the number of columns if
>> you
>>>>> use a format that allows empty headers and contains null column names.
>>>>>
>>>>>
>>>>> The ultimate result is that we should document that the purpose of the
>>>>> header names is to provide a list of non-null header names in the order
>>>>> they occur in the header and thus represent keys that can be used in
>> the
>>>>> header map. In certain circumstances there may be more columns in the
>> data
>>>>> than there are header names.
>>>>>
>>>>>
>>>>> Alex
>>>>>
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/CSV-248 <
>>>>> https://issues.apache.org/jira/browse/CSV-248>
>>>>>
>>>>> [2]
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>>>>> <
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>>>>>>
>>>>>
>>>>> [3] https://markmail.org/message/woti2iymecosihx6 <
>>>>> https://markmail.org/message/woti2iymecosihx6>
>>>>>
>>>>>
>>>>>
>>>>>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>>>>>>
>>>>>> We have fixed quite a few bugs and added some significant enhancements
>>>>>> since Apache Commons CSV 1.7 was released, so I would like to release
>>>>>> Apache Commons CSV 1.8.
>>>>>>
>>>>>> Apache Commons CSV 1.8 RC1 is available for review here:
>>>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
>>>>>> revision 37670)
>>>>>>
>>>>>> The Git tag commons-csv-1.8-RC1 commit for this RC is
>>>>>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>>>>>>
>>>>>>
>>>>>
>> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
>>>>>> You may checkout this tag using:
>>>>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>>>>> --branch
>>>>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>>>>
>>>>>> Maven artifacts are here:
>>>>>>
>>>>>>
>>>>>
>> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>>>>>>
>>>>>> These are the artifacts and their hashes:
>>>>>>
>>>>>> #Release SHA-512s
>>>>>> #Sat Jan 18 12:01:01 EST 2020
>>>>>>
>>>>>
>> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
>>>>>>
>>>>>
>> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
>>>>>>
>>>>>
>> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
>>>>>>
>>>>>
>> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
>>>>>>
>>>>>
>> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
>>>>>>
>>>>>
>> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
>>>>>>
>>>>>
>> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
>>>>>>
>>>>>
>> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>>>>>>
>>>>>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
>>>>>> japicmp clean package site deploy' using:
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime:
>> C:\Program
>>>>>> Files\Java\jdk1.8.0_241\jre
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> Additional tests with 'mvn -V clean test' using:
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime:
>> C:\Program
>>>>>> Files\Java\jdk1.8.0_241\jre
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
>>>>>> Files\Java\jdk-11.0.6
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>>>>> Files\Java\jdk-12.0.2
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>>>>> Files\Java\jdk-13.0.2
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
>>>>>> Files\Java\openjdk\jdk-14
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> JaCoCo fails on Java 15-EA because it does not know about class file
>>>>> major
>>>>>> version 59:
>>>>>>
>>>>>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
>>>>> major
>>>>>> version 59
>>>>>> at
>>>>>>
>>>>>
>> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>>>>>>
>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
>>>>>> Files\Java\openjdk\jdk-15
>>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>> "windows"
>>>>>>
>>>>>> Details of changes since 1.7 are in the release notes:
>>>>>>
>>>>>>
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>>>>>>
>>>>>>
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>>>>>>
>>>>>> Site:
>>>>>>
>>>>>>
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
>>>>>> (note some *relative* links are broken and the 1.8 directories are
>>>>> not
>>>>>> yet created - these will be OK once the site is deployed.)
>>>>>>
>>>>>> JApiCmp Report (compared to 1.7):
>>>>>>
>>>>>>
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>>>>>>
>>>>>> RAT Report:
>>>>>>
>>>>>>
>>>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>>>>>>
>>>>>> KEYS:
>>>>>> https://www.apache.org/dist/commons/KEYS
>>>>>>
>>>>>> Please review the release candidate and vote.
>>>>>> This vote will close no sooner that 72 hours from now.
>>>>>>
>>>>>> [ ] +1 Release these artifacts
>>>>>> [ ] +0 OK, but...
>>>>>> [ ] -0 OK, but really should fix...
>>>>>> [ ] -1 I oppose this release because...
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Gary Gregory,
>>>>>> Release Manager (using key 86fdc7e2a11262cb)
>>>>>>
>>>>>> For following is intended as a helper and refresher for reviewers.
>>>>>>
>>>>>> Validating a release candidate
>>>>>> ==============================
>>>>>>
>>>>>> These guidelines are NOT complete.
>>>>>>
>>>>>> Requirements: Git, Java, Maven.
>>>>>>
>>>>>> You can validate a release from a release candidate (RC) tag as
>> follows.
>>>>>>
>>>>>> 1) Clone and checkout the RC tag
>>>>>>
>>>>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>> --branch
>>>>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>>>> cd commons-csv-1.8-RC1
>>>>>>
>>>>>> 2) Check Apache licenses
>>>>>>
>>>>>> This step is not required if the site includes a RAT report page which
>>>>> you
>>>>>> then must check.
>>>>>>
>>>>>> mvn apache-rat:check
>>>>>>
>>>>>> 3) Check binary compatibility
>>>>>>
>>>>>> Older components still use Apache Clirr:
>>>>>>
>>>>>> This step is not required if the site includes a Clirr report page
>> which
>>>>>> you then must check.
>>>>>>
>>>>>> mvn clirr:check
>>>>>>
>>>>>> Newer components use JApiCmp with the japicmp Maven Profile:
>>>>>>
>>>>>> This step is not required if the site includes a JApiCmp report page
>>>>> which
>>>>>> you then must check.
>>>>>>
>>>>>> mvn install -DskipTests -P japicmp japicmp:cmp
>>>>>>
>>>>>> 4) Build the package
>>>>>>
>>>>>> mvn -V clean package
>>>>>>
>>>>>> You can record the Maven and Java version produced by -V in your VOTE
>>>>> reply.
>>>>>> To gather OS information from a command line:
>>>>>> Windows: ver
>>>>>> Linux: uname -a
>>>>>>
>>>>>> 5) Build the site for a single module project
>>>>>>
>>>>>> Note: Some plugins require the components to be installed instead of
>>>>>> packaged.
>>>>>>
>>>>>> mvn site
>>>>>> Check the site reports in:
>>>>>> - Windows: target\site\index.html
>>>>>> - Linux: target/site/index.html
>>>>>>
>>>>>> 6) Build the site for a multi-module project
>>>>>>
>>>>>> mvn site
>>>>>> mvn site:stage
>>>>>> Check the site reports in:
>>>>>> - Windows: target\site\index.html
>>>>>> - Linux: target/site/index.html
>>>>>>
>>>>>> -the end-
>>>>>
>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 20, 2020 at 6:43 PM Rob Tompkins <ch...@gmail.com> wrote:
>
>
> > On Jan 20, 2020, at 6:41 PM, Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > On Mon, Jan 20, 2020 at 6:28 PM Gary Gregory <ga...@gmail.com>
> wrote:
> >
> >>> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <alex.d.herbert@gmail.com
> >
> >>> wrote:
> >>>
> >>> Hi Gary,
> >>>
> >>> I raised a few niggles a while back with CSV and the discussion did not
> >>> receive a response on how to proceed.
> >>>
> >>> There is the major bug CSV-248 where the CSVRecord is not Serializable
> >>> [1]. This requires a decision on what to do to fix it. This bug is
> still
> >>> present in 1.8 RC1 as found by FindBugs [2].
> >>>
> >>> From what I can see the CSVRecord maintains a reference to the
> CSVParser.
> >>> This chain of objects maintained in memory is not serializable and
> leads
> >>> back to the original input Reader.
> >>>
> >>> I can see from the JApiCmp report that the serial version id was
> changed
> >>> for CSVRecord this release so there is still an intention to support
> >>> serialization. So this should be a blocker.
> >>>
> >>> I could not find a serialisation test in the unit tests for CSVRecord.
> >>> This quick test added to CSVRecordTest fails:
> >>>
> >>>
> >>> @Test
> >>> public void testSerialization() throws IOException {
> >>> CSVRecord shortRec;
> >>> try (final CSVParser parser = CSVParser.parse("a,b",
> >>> CSVFormat.newFormat(','))) {
> >>> shortRec = parser.iterator().next();
> >>> }
> >>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
> >>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> >>> oos.writeObject(shortRec);
> >>> }
> >>> }
> >>>
> >>> mvn test -Dtest=CSVRecordTest
> >>>
> >>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> >>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> >>> at
> >>>
> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
> >>>
> >>> If I mark the field csvParser as transient it passes. So this is a
> >>> problem as raised by FindBugs.
> >>>
> >>
> >> Making the field transient would indeed fix this test but... some of the
> >> record APIs would then fail with NPEs... so we would be kicking the can
> >> down the road basically. Making the parser serializable is going in the
> >> wrong direction feature-wise IMO, so let's not go there. A serialization
> >> proxy would be less worse but should we provide such a thing? I would
> say
> >> no. I am OK with making the field transient despite the can kicking, so
> >> let's do that.
> >>
> >
> > I suppose we should bump the serialVersionUID...
>
> +1 to that
>
The pickle if we do that is that we will be binary incompatible as JApiCmp
notes:
MODIFIED (Serializable incompatible(!): serialVersionUID modified) final
public class org.apache.commons.csv.CSVRecord
Gary
>
>
> > Gary
> >
> >
> >>
> >> Gary
> >>
> >>
> >>>
> >>>
> >>> I also raised [3] the strange implementation of the CSVParser
> >>> getHeaderNames() which ignores null headers as they cannot be used as
> a key
> >>> into the map. However the list of column names could contain the null
> >>> values. This test currently fails:
> >>>
> >>> @Test
> >>> public void testHeaderNamesWithNull() throws IOException {
> >>> final Reader in = new
> >>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
> >>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
> >>>
> >>> .withNullString("null")
> >>>
> >>> .withAllowMissingColumnNames()
> >>>
> >>> .parse(in).iterator();
> >>> final CSVRecord record = records.next();
> >>> assertEquals(Arrays.asList("header1", null, "header3"),
> >>> record.getParser().getHeaderNames());
> >>> }
> >>>
> >>> I am not saying it should pass but at least the documentation should
> >>> state the behaviour in this edge case. That is the list of header
> names may
> >>> be shorter than the number of columns when the parser is configured to
> >>> allow null headers. I’ve not raised a bug ticket for this as it is
> open to
> >>> opinion if this is by design or actually a bug. This issue is still
> present
> >>> in 1.8 RC1.
> >>>
> >>> Previously I suggested documentation changes for this and another edge
> >>> case using the header map to be added to the javadoc for
> getHeaderNames()
> >>> and getHeaderMap():
> >>>
> >>> - Documentation:
> >>>
> >>> The mapping is only guaranteed to be a one-to-one mapping if the record
> >>> was created with a format that does not allow duplicate or null header
> >>> names. Null headers are excluded from the map and duplicates can only
> map
> >>> to 1 column.
> >>>
> >>>
> >>> - Bug / Documentation
> >>>
> >>> The CSVParser only stores headers names in a list of header names if
> they
> >>> are not null. So the list can be shorter than the number of columns if
> you
> >>> use a format that allows empty headers and contains null column names.
> >>>
> >>>
> >>> The ultimate result is that we should document that the purpose of the
> >>> header names is to provide a list of non-null header names in the order
> >>> they occur in the header and thus represent keys that can be used in
> the
> >>> header map. In certain circumstances there may be more columns in the
> data
> >>> than there are header names.
> >>>
> >>>
> >>> Alex
> >>>
> >>>
> >>> [1] https://issues.apache.org/jira/browse/CSV-248 <
> >>> https://issues.apache.org/jira/browse/CSV-248>
> >>>
> >>> [2]
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >>> <
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >>>>
> >>>
> >>> [3] https://markmail.org/message/woti2iymecosihx6 <
> >>> https://markmail.org/message/woti2iymecosihx6>
> >>>
> >>>
> >>>
> >>>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> >>>>
> >>>> We have fixed quite a few bugs and added some significant enhancements
> >>>> since Apache Commons CSV 1.7 was released, so I would like to release
> >>>> Apache Commons CSV 1.8.
> >>>>
> >>>> Apache Commons CSV 1.8 RC1 is available for review here:
> >>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> >>>> revision 37670)
> >>>>
> >>>> The Git tag commons-csv-1.8-RC1 commit for this RC is
> >>>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> >>>>
> >>>>
> >>>
> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> >>>> You may checkout this tag using:
> >>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> >>> --branch
> >>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >>>>
> >>>> Maven artifacts are here:
> >>>>
> >>>>
> >>>
> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> >>>>
> >>>> These are the artifacts and their hashes:
> >>>>
> >>>> #Release SHA-512s
> >>>> #Sat Jan 18 12:01:01 EST 2020
> >>>>
> >>>
> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> >>>>
> >>>
> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> >>>>
> >>>
> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> >>>>
> >>>
> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> >>>>
> >>>
> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> >>>>
> >>>
> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> >>>>
> >>>
> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> >>>>
> >>>
> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> >>>>
> >>>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> >>>> japicmp clean package site deploy' using:
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime:
> C:\Program
> >>>> Files\Java\jdk1.8.0_241\jre
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> Additional tests with 'mvn -V clean test' using:
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime:
> C:\Program
> >>>> Files\Java\jdk1.8.0_241\jre
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> >>>> Files\Java\jdk-11.0.6
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> >>>> Files\Java\jdk-12.0.2
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> >>>> Files\Java\jdk-13.0.2
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> >>>> Files\Java\openjdk\jdk-14
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> JaCoCo fails on Java 15-EA because it does not know about class file
> >>> major
> >>>> version 59:
> >>>>
> >>>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
> >>> major
> >>>> version 59
> >>>> at
> >>>>
> >>>
> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> >>>>
> >>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> >>>> Files\Java\openjdk\jdk-15
> >>>> Default locale: en_US, platform encoding: Cp1252
> >>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> >>>>
> >>>> Details of changes since 1.7 are in the release notes:
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> >>>>
> >>>> Site:
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> >>>> (note some *relative* links are broken and the 1.8 directories are
> >>> not
> >>>> yet created - these will be OK once the site is deployed.)
> >>>>
> >>>> JApiCmp Report (compared to 1.7):
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> >>>>
> >>>> RAT Report:
> >>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> >>>>
> >>>> KEYS:
> >>>> https://www.apache.org/dist/commons/KEYS
> >>>>
> >>>> Please review the release candidate and vote.
> >>>> This vote will close no sooner that 72 hours from now.
> >>>>
> >>>> [ ] +1 Release these artifacts
> >>>> [ ] +0 OK, but...
> >>>> [ ] -0 OK, but really should fix...
> >>>> [ ] -1 I oppose this release because...
> >>>>
> >>>> Thank you,
> >>>>
> >>>> Gary Gregory,
> >>>> Release Manager (using key 86fdc7e2a11262cb)
> >>>>
> >>>> For following is intended as a helper and refresher for reviewers.
> >>>>
> >>>> Validating a release candidate
> >>>> ==============================
> >>>>
> >>>> These guidelines are NOT complete.
> >>>>
> >>>> Requirements: Git, Java, Maven.
> >>>>
> >>>> You can validate a release from a release candidate (RC) tag as
> follows.
> >>>>
> >>>> 1) Clone and checkout the RC tag
> >>>>
> >>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> --branch
> >>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >>>> cd commons-csv-1.8-RC1
> >>>>
> >>>> 2) Check Apache licenses
> >>>>
> >>>> This step is not required if the site includes a RAT report page which
> >>> you
> >>>> then must check.
> >>>>
> >>>> mvn apache-rat:check
> >>>>
> >>>> 3) Check binary compatibility
> >>>>
> >>>> Older components still use Apache Clirr:
> >>>>
> >>>> This step is not required if the site includes a Clirr report page
> which
> >>>> you then must check.
> >>>>
> >>>> mvn clirr:check
> >>>>
> >>>> Newer components use JApiCmp with the japicmp Maven Profile:
> >>>>
> >>>> This step is not required if the site includes a JApiCmp report page
> >>> which
> >>>> you then must check.
> >>>>
> >>>> mvn install -DskipTests -P japicmp japicmp:cmp
> >>>>
> >>>> 4) Build the package
> >>>>
> >>>> mvn -V clean package
> >>>>
> >>>> You can record the Maven and Java version produced by -V in your VOTE
> >>> reply.
> >>>> To gather OS information from a command line:
> >>>> Windows: ver
> >>>> Linux: uname -a
> >>>>
> >>>> 5) Build the site for a single module project
> >>>>
> >>>> Note: Some plugins require the components to be installed instead of
> >>>> packaged.
> >>>>
> >>>> mvn site
> >>>> Check the site reports in:
> >>>> - Windows: target\site\index.html
> >>>> - Linux: target/site/index.html
> >>>>
> >>>> 6) Build the site for a multi-module project
> >>>>
> >>>> mvn site
> >>>> mvn site:stage
> >>>> Check the site reports in:
> >>>> - Windows: target\site\index.html
> >>>> - Linux: target/site/index.html
> >>>>
> >>>> -the end-
> >>>
> >>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Rob Tompkins <ch...@gmail.com>.
> On Jan 20, 2020, at 6:41 PM, Gary Gregory <ga...@gmail.com> wrote:
>
> On Mon, Jan 20, 2020 at 6:28 PM Gary Gregory <ga...@gmail.com> wrote:
>
>>> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
>>> wrote:
>>>
>>> Hi Gary,
>>>
>>> I raised a few niggles a while back with CSV and the discussion did not
>>> receive a response on how to proceed.
>>>
>>> There is the major bug CSV-248 where the CSVRecord is not Serializable
>>> [1]. This requires a decision on what to do to fix it. This bug is still
>>> present in 1.8 RC1 as found by FindBugs [2].
>>>
>>> From what I can see the CSVRecord maintains a reference to the CSVParser.
>>> This chain of objects maintained in memory is not serializable and leads
>>> back to the original input Reader.
>>>
>>> I can see from the JApiCmp report that the serial version id was changed
>>> for CSVRecord this release so there is still an intention to support
>>> serialization. So this should be a blocker.
>>>
>>> I could not find a serialisation test in the unit tests for CSVRecord.
>>> This quick test added to CSVRecordTest fails:
>>>
>>>
>>> @Test
>>> public void testSerialization() throws IOException {
>>> CSVRecord shortRec;
>>> try (final CSVParser parser = CSVParser.parse("a,b",
>>> CSVFormat.newFormat(','))) {
>>> shortRec = parser.iterator().next();
>>> }
>>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
>>> oos.writeObject(shortRec);
>>> }
>>> }
>>>
>>> mvn test -Dtest=CSVRecordTest
>>>
>>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
>>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
>>> at
>>> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>>>
>>> If I mark the field csvParser as transient it passes. So this is a
>>> problem as raised by FindBugs.
>>>
>>
>> Making the field transient would indeed fix this test but... some of the
>> record APIs would then fail with NPEs... so we would be kicking the can
>> down the road basically. Making the parser serializable is going in the
>> wrong direction feature-wise IMO, so let's not go there. A serialization
>> proxy would be less worse but should we provide such a thing? I would say
>> no. I am OK with making the field transient despite the can kicking, so
>> let's do that.
>>
>
> I suppose we should bump the serialVersionUID...
+1 to that
> Gary
>
>
>>
>> Gary
>>
>>
>>>
>>>
>>> I also raised [3] the strange implementation of the CSVParser
>>> getHeaderNames() which ignores null headers as they cannot be used as a key
>>> into the map. However the list of column names could contain the null
>>> values. This test currently fails:
>>>
>>> @Test
>>> public void testHeaderNamesWithNull() throws IOException {
>>> final Reader in = new
>>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
>>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>>>
>>> .withNullString("null")
>>>
>>> .withAllowMissingColumnNames()
>>>
>>> .parse(in).iterator();
>>> final CSVRecord record = records.next();
>>> assertEquals(Arrays.asList("header1", null, "header3"),
>>> record.getParser().getHeaderNames());
>>> }
>>>
>>> I am not saying it should pass but at least the documentation should
>>> state the behaviour in this edge case. That is the list of header names may
>>> be shorter than the number of columns when the parser is configured to
>>> allow null headers. I’ve not raised a bug ticket for this as it is open to
>>> opinion if this is by design or actually a bug. This issue is still present
>>> in 1.8 RC1.
>>>
>>> Previously I suggested documentation changes for this and another edge
>>> case using the header map to be added to the javadoc for getHeaderNames()
>>> and getHeaderMap():
>>>
>>> - Documentation:
>>>
>>> The mapping is only guaranteed to be a one-to-one mapping if the record
>>> was created with a format that does not allow duplicate or null header
>>> names. Null headers are excluded from the map and duplicates can only map
>>> to 1 column.
>>>
>>>
>>> - Bug / Documentation
>>>
>>> The CSVParser only stores headers names in a list of header names if they
>>> are not null. So the list can be shorter than the number of columns if you
>>> use a format that allows empty headers and contains null column names.
>>>
>>>
>>> The ultimate result is that we should document that the purpose of the
>>> header names is to provide a list of non-null header names in the order
>>> they occur in the header and thus represent keys that can be used in the
>>> header map. In certain circumstances there may be more columns in the data
>>> than there are header names.
>>>
>>>
>>> Alex
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/CSV-248 <
>>> https://issues.apache.org/jira/browse/CSV-248>
>>>
>>> [2]
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>>> <
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>>>>
>>>
>>> [3] https://markmail.org/message/woti2iymecosihx6 <
>>> https://markmail.org/message/woti2iymecosihx6>
>>>
>>>
>>>
>>>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>>>>
>>>> We have fixed quite a few bugs and added some significant enhancements
>>>> since Apache Commons CSV 1.7 was released, so I would like to release
>>>> Apache Commons CSV 1.8.
>>>>
>>>> Apache Commons CSV 1.8 RC1 is available for review here:
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
>>>> revision 37670)
>>>>
>>>> The Git tag commons-csv-1.8-RC1 commit for this RC is
>>>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>>>>
>>>>
>>> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
>>>> You may checkout this tag using:
>>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>>> --branch
>>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>>
>>>> Maven artifacts are here:
>>>>
>>>>
>>> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>>>>
>>>> These are the artifacts and their hashes:
>>>>
>>>> #Release SHA-512s
>>>> #Sat Jan 18 12:01:01 EST 2020
>>>>
>>> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
>>>>
>>> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
>>>>
>>> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
>>>>
>>> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
>>>>
>>> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
>>>>
>>> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
>>>>
>>> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
>>>>
>>> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>>>>
>>>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
>>>> japicmp clean package site deploy' using:
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\jdk1.8.0_241\jre
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> Additional tests with 'mvn -V clean test' using:
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\jdk1.8.0_241\jre
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\jdk-11.0.6
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\jdk-12.0.2
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\jdk-13.0.2
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\openjdk\jdk-14
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> JaCoCo fails on Java 15-EA because it does not know about class file
>>> major
>>>> version 59:
>>>>
>>>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
>>> major
>>>> version 59
>>>> at
>>>>
>>> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>>>>
>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
>>>> Files\Java\openjdk\jdk-15
>>>> Default locale: en_US, platform encoding: Cp1252
>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>
>>>> Details of changes since 1.7 are in the release notes:
>>>>
>>>>
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>>>>
>>>>
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>>>>
>>>> Site:
>>>>
>>>>
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
>>>> (note some *relative* links are broken and the 1.8 directories are
>>> not
>>>> yet created - these will be OK once the site is deployed.)
>>>>
>>>> JApiCmp Report (compared to 1.7):
>>>>
>>>>
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>>>>
>>>> RAT Report:
>>>>
>>>>
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>>>>
>>>> KEYS:
>>>> https://www.apache.org/dist/commons/KEYS
>>>>
>>>> Please review the release candidate and vote.
>>>> This vote will close no sooner that 72 hours from now.
>>>>
>>>> [ ] +1 Release these artifacts
>>>> [ ] +0 OK, but...
>>>> [ ] -0 OK, but really should fix...
>>>> [ ] -1 I oppose this release because...
>>>>
>>>> Thank you,
>>>>
>>>> Gary Gregory,
>>>> Release Manager (using key 86fdc7e2a11262cb)
>>>>
>>>> For following is intended as a helper and refresher for reviewers.
>>>>
>>>> Validating a release candidate
>>>> ==============================
>>>>
>>>> These guidelines are NOT complete.
>>>>
>>>> Requirements: Git, Java, Maven.
>>>>
>>>> You can validate a release from a release candidate (RC) tag as follows.
>>>>
>>>> 1) Clone and checkout the RC tag
>>>>
>>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
>>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>> cd commons-csv-1.8-RC1
>>>>
>>>> 2) Check Apache licenses
>>>>
>>>> This step is not required if the site includes a RAT report page which
>>> you
>>>> then must check.
>>>>
>>>> mvn apache-rat:check
>>>>
>>>> 3) Check binary compatibility
>>>>
>>>> Older components still use Apache Clirr:
>>>>
>>>> This step is not required if the site includes a Clirr report page which
>>>> you then must check.
>>>>
>>>> mvn clirr:check
>>>>
>>>> Newer components use JApiCmp with the japicmp Maven Profile:
>>>>
>>>> This step is not required if the site includes a JApiCmp report page
>>> which
>>>> you then must check.
>>>>
>>>> mvn install -DskipTests -P japicmp japicmp:cmp
>>>>
>>>> 4) Build the package
>>>>
>>>> mvn -V clean package
>>>>
>>>> You can record the Maven and Java version produced by -V in your VOTE
>>> reply.
>>>> To gather OS information from a command line:
>>>> Windows: ver
>>>> Linux: uname -a
>>>>
>>>> 5) Build the site for a single module project
>>>>
>>>> Note: Some plugins require the components to be installed instead of
>>>> packaged.
>>>>
>>>> mvn site
>>>> Check the site reports in:
>>>> - Windows: target\site\index.html
>>>> - Linux: target/site/index.html
>>>>
>>>> 6) Build the site for a multi-module project
>>>>
>>>> mvn site
>>>> mvn site:stage
>>>> Check the site reports in:
>>>> - Windows: target\site\index.html
>>>> - Linux: target/site/index.html
>>>>
>>>> -the end-
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 20, 2020 at 6:28 PM Gary Gregory <ga...@gmail.com> wrote:
> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
> wrote:
>
>> Hi Gary,
>>
>> I raised a few niggles a while back with CSV and the discussion did not
>> receive a response on how to proceed.
>>
>> There is the major bug CSV-248 where the CSVRecord is not Serializable
>> [1]. This requires a decision on what to do to fix it. This bug is still
>> present in 1.8 RC1 as found by FindBugs [2].
>>
>> From what I can see the CSVRecord maintains a reference to the CSVParser.
>> This chain of objects maintained in memory is not serializable and leads
>> back to the original input Reader.
>>
>> I can see from the JApiCmp report that the serial version id was changed
>> for CSVRecord this release so there is still an intention to support
>> serialization. So this should be a blocker.
>>
>> I could not find a serialisation test in the unit tests for CSVRecord.
>> This quick test added to CSVRecordTest fails:
>>
>>
>> @Test
>> public void testSerialization() throws IOException {
>> CSVRecord shortRec;
>> try (final CSVParser parser = CSVParser.parse("a,b",
>> CSVFormat.newFormat(','))) {
>> shortRec = parser.iterator().next();
>> }
>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
>> oos.writeObject(shortRec);
>> }
>> }
>>
>> mvn test -Dtest=CSVRecordTest
>>
>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
>> at
>> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>>
>> If I mark the field csvParser as transient it passes. So this is a
>> problem as raised by FindBugs.
>>
>
> Making the field transient would indeed fix this test but... some of the
> record APIs would then fail with NPEs... so we would be kicking the can
> down the road basically. Making the parser serializable is going in the
> wrong direction feature-wise IMO, so let's not go there. A serialization
> proxy would be less worse but should we provide such a thing? I would say
> no. I am OK with making the field transient despite the can kicking, so
> let's do that.
>
I suppose we should bump the serialVersionUID...
Gary
>
> Gary
>
>
>>
>>
>> I also raised [3] the strange implementation of the CSVParser
>> getHeaderNames() which ignores null headers as they cannot be used as a key
>> into the map. However the list of column names could contain the null
>> values. This test currently fails:
>>
>> @Test
>> public void testHeaderNamesWithNull() throws IOException {
>> final Reader in = new
>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>>
>> .withNullString("null")
>>
>> .withAllowMissingColumnNames()
>>
>> .parse(in).iterator();
>> final CSVRecord record = records.next();
>> assertEquals(Arrays.asList("header1", null, "header3"),
>> record.getParser().getHeaderNames());
>> }
>>
>> I am not saying it should pass but at least the documentation should
>> state the behaviour in this edge case. That is the list of header names may
>> be shorter than the number of columns when the parser is configured to
>> allow null headers. I’ve not raised a bug ticket for this as it is open to
>> opinion if this is by design or actually a bug. This issue is still present
>> in 1.8 RC1.
>>
>> Previously I suggested documentation changes for this and another edge
>> case using the header map to be added to the javadoc for getHeaderNames()
>> and getHeaderMap():
>>
>> - Documentation:
>>
>> The mapping is only guaranteed to be a one-to-one mapping if the record
>> was created with a format that does not allow duplicate or null header
>> names. Null headers are excluded from the map and duplicates can only map
>> to 1 column.
>>
>>
>> - Bug / Documentation
>>
>> The CSVParser only stores headers names in a list of header names if they
>> are not null. So the list can be shorter than the number of columns if you
>> use a format that allows empty headers and contains null column names.
>>
>>
>> The ultimate result is that we should document that the purpose of the
>> header names is to provide a list of non-null header names in the order
>> they occur in the header and thus represent keys that can be used in the
>> header map. In certain circumstances there may be more columns in the data
>> than there are header names.
>>
>>
>> Alex
>>
>>
>> [1] https://issues.apache.org/jira/browse/CSV-248 <
>> https://issues.apache.org/jira/browse/CSV-248>
>>
>> [2]
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>> <
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>> >
>>
>> [3] https://markmail.org/message/woti2iymecosihx6 <
>> https://markmail.org/message/woti2iymecosihx6>
>>
>>
>>
>> > On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>> >
>> > We have fixed quite a few bugs and added some significant enhancements
>> > since Apache Commons CSV 1.7 was released, so I would like to release
>> > Apache Commons CSV 1.8.
>> >
>> > Apache Commons CSV 1.8 RC1 is available for review here:
>> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
>> > revision 37670)
>> >
>> > The Git tag commons-csv-1.8-RC1 commit for this RC is
>> > c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>> >
>> >
>> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
>> > You may checkout this tag using:
>> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>> --branch
>> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
>> >
>> > Maven artifacts are here:
>> >
>> >
>> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>> >
>> > These are the artifacts and their hashes:
>> >
>> > #Release SHA-512s
>> > #Sat Jan 18 12:01:01 EST 2020
>> >
>> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
>> >
>> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
>> >
>> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
>> >
>> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
>> >
>> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
>> >
>> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
>> >
>> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
>> >
>> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>> >
>> > I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
>> > japicmp clean package site deploy' using:
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\jdk1.8.0_241\jre
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > Additional tests with 'mvn -V clean test' using:
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\jdk1.8.0_241\jre
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\jdk-11.0.6
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\jdk-12.0.2
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\jdk-13.0.2
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\openjdk\jdk-14
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > JaCoCo fails on Java 15-EA because it does not know about class file
>> major
>> > version 59:
>> >
>> > Caused by: java.lang.IllegalArgumentException: Unsupported class file
>> major
>> > version 59
>> > at
>> >
>> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>> >
>> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
>> > Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
>> > Files\Java\openjdk\jdk-15
>> > Default locale: en_US, platform encoding: Cp1252
>> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>> >
>> > Details of changes since 1.7 are in the release notes:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>> >
>> > Site:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
>> > (note some *relative* links are broken and the 1.8 directories are
>> not
>> > yet created - these will be OK once the site is deployed.)
>> >
>> > JApiCmp Report (compared to 1.7):
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>> >
>> > RAT Report:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>> >
>> > KEYS:
>> > https://www.apache.org/dist/commons/KEYS
>> >
>> > Please review the release candidate and vote.
>> > This vote will close no sooner that 72 hours from now.
>> >
>> > [ ] +1 Release these artifacts
>> > [ ] +0 OK, but...
>> > [ ] -0 OK, but really should fix...
>> > [ ] -1 I oppose this release because...
>> >
>> > Thank you,
>> >
>> > Gary Gregory,
>> > Release Manager (using key 86fdc7e2a11262cb)
>> >
>> > For following is intended as a helper and refresher for reviewers.
>> >
>> > Validating a release candidate
>> > ==============================
>> >
>> > These guidelines are NOT complete.
>> >
>> > Requirements: Git, Java, Maven.
>> >
>> > You can validate a release from a release candidate (RC) tag as follows.
>> >
>> > 1) Clone and checkout the RC tag
>> >
>> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
>> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
>> > cd commons-csv-1.8-RC1
>> >
>> > 2) Check Apache licenses
>> >
>> > This step is not required if the site includes a RAT report page which
>> you
>> > then must check.
>> >
>> > mvn apache-rat:check
>> >
>> > 3) Check binary compatibility
>> >
>> > Older components still use Apache Clirr:
>> >
>> > This step is not required if the site includes a Clirr report page which
>> > you then must check.
>> >
>> > mvn clirr:check
>> >
>> > Newer components use JApiCmp with the japicmp Maven Profile:
>> >
>> > This step is not required if the site includes a JApiCmp report page
>> which
>> > you then must check.
>> >
>> > mvn install -DskipTests -P japicmp japicmp:cmp
>> >
>> > 4) Build the package
>> >
>> > mvn -V clean package
>> >
>> > You can record the Maven and Java version produced by -V in your VOTE
>> reply.
>> > To gather OS information from a command line:
>> > Windows: ver
>> > Linux: uname -a
>> >
>> > 5) Build the site for a single module project
>> >
>> > Note: Some plugins require the components to be installed instead of
>> > packaged.
>> >
>> > mvn site
>> > Check the site reports in:
>> > - Windows: target\site\index.html
>> > - Linux: target/site/index.html
>> >
>> > 6) Build the site for a multi-module project
>> >
>> > mvn site
>> > mvn site:stage
>> > Check the site reports in:
>> > - Windows: target\site\index.html
>> > - Linux: target/site/index.html
>> >
>> > -the end-
>>
>>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
I'll should be able to create RC2 at the weekend or sooner.
Gary
On Tue, Jan 28, 2020 at 10:28 AM Alex Herbert <al...@gmail.com>
wrote:
>
> On 28/01/2020 15:15, Gary Gregory wrote:
> > On Mon, Jan 27, 2020 at 7:58 PM Gary Gregory <ga...@gmail.com>
> wrote:
> >
> >> On Mon, Jan 27, 2020 at 6:54 PM Alex Herbert <al...@gmail.com>
> >> wrote:
> >>
> >>> I’ve had a look at the Serialization of CSVRecord. Fields have been
> added
> >>> and removed from CSVRecord as:
> >>>
> >>> 1.0
> >>>
> >>> /** The accumulated comments (if any) */
> >>> private final String comment;
> >>>
> >>> /** The column name to index mapping. */
> >>> private final Map<String, Integer> mapping;
> >>>
> >>> /** The record number. */
> >>> private final long recordNumber;
> >>>
> >>> /** The values of the record */
> >>> private final String[] values;
> >>>
> >>> 1.1
> >>>
> >>> // Added
> >>>
> >>> private final long characterPosition;
> >>>
> >>> 1.2 to 1.6
> >>>
> >>> No changes
> >>>
> >>> 1.7
> >>>
> >>> // Removed
> >>>
> >>> /** The column name to index mapping. */
> >>> private final Map<String, Integer> mapping;
> >>>
> >>> // Added a non-serialisable field -> broken
> >>>
> >>> /** The parser that originates this record. */
> >>> private final CSVParser parser;
> >>>
> >>> 1.8
> >>>
> >>> // Fixed by marking as transient
> >>> private final transient CSVParser parser;
> >>>
> >>>
> >>> If you write and read records in 1.8 the parser is not serialised. This
> >>> contains the map between column names and indices. So the following
> methods
> >>> are affected:
> >>>
> >>> boolean CSVRecord.isMapped(String);
> >>> boolean CSVRecord.isSet(String);
> >>> Map<String, String> CSVRecord.toMap();
> >>> String CSVRecord.get(String);
> >>>
> >>> The original object would have valid returns for these. A deserialised
> >>> version returns false for all map keys, the map representation is an
> empty
> >>> map and get will throw as access by name is not supported.
> >>>
> >>>
> >>> You can read records from 1.0 in using 1.8. It ignores the map that was
> >>> serialised as part of the record. It also ignore the missing character
> >>> position field and assigns it a default value of 0.
> >>>
> >>>
> >>> Likewise you can read records from 1.8 in using 1.0. Here the map is
> >>> missing and so all functionality linked to the map fails. The failures
> are
> >>> exactly as for reading 1.0 in to 1.8.
> >>>
> >>>
> >>> So this indicates you can serialise and deserialise between different
> >>> versions back to 1.0 excluding 1.7. Deserialisation may not create
> records
> >>> entirely. The following lists what should work with no loss of
> >>> functionality in the destination version:
> >>>
> >>> Serialised from => deserialised to
> >>>
> >>> 1.0 => 1.0
> >>> 1.1 - 1.6 => 1.0 (the character position field is ignored)
> >>> 1.1 - 1.6 => 1.1 - 1.6
> >>>
> >>> The following will result in absent information:
> >>>
> >>> 1.0 => 1.1 - 1.6 (no character position)
> >>> 1.0 => 1.8 (no character position, no mapping functionality)
> >>> 1.1 - 1.6 => 1.8 (no mapping functionality)
> >>> 1.8 => 1.0 - 1.8 (no mapping functionality)
> >>>
> >>> 1.7 is ignored as it cannot be serialised
> >>>
> >>>
> >>> So even though you can deserialise between versions without an
> exception
> >>> the functionality is impaired in certain cases, i.e. when jumping
> between
> >>> 1.0-1.6 and 1.8. Since deserialisation can create an instance a change
> to
> >>> the serial version ID is not warranted. The compatibility should be
> noted
> >>> in the documentation, e.g. something like:
> >>>
> >>> * <p>
> >>> * Note: Support for {@link Serializable} is scheduled to be removed
> in
> >>> version 2.0.
> >>> * In version 1.8 the mapping between the column header and the column
> >>> index was
> >>> * removed from the serialised state. The class maintains
> serialization
> >>> compatibility
> >>> * with versions pre-1.8 for the record values; these must be
> accessed by
> >>> index
> >>> * following deserialization. There will be loss of any functionally
> >>> linked to the header
> >>> * map when transferring serialised forms pre-1.8 to 1.8 and vice
> versa.
> >>> * </p>
> >>>
> >>> Thoughts on this?
> >>>
> >> +1 to updating the docs.
> >>
> > Thanks for the update Alex. Do we want anything else for RC2?
>
> Not from me. Serialization is fixed (for compatibility) and the docs now
> state it won't be fully functional going forward. I hope that is the end
> of that.
>
> > Gary
> >
> >
> >> Gary
> >>
> >>
> >>> Alex
> >>>
> >>>> On 25 Jan 2020, at 23:02, Alex Herbert <al...@gmail.com>
> >>> wrote:
> >>>>
> >>>>
> >>>>> On 25 Jan 2020, at 22:05, Gary Gregory <ga...@gmail.com>
> wrote:
> >>>>>
> >>>>> Hi Alex,
> >>>>>
> >>>>> Is there more you'd like to do for 1.8?
> >>>>>
> >>>> Not functionality. The remaining things are the documentation of:
> >>>>
> >>>> - the intent to remove Serialisation support to the CSVRecord in 2.0
> >>>> - the intent to not support Serialisation of additional fields added
> >>> after from 1.6
> >>>> This is for the CSVRecord class header and to the release notes in
> >>> changes.xml.
> >>>> We have the test to show that you can deserialise from 1.6. I think
> >>> that I should change the .bin serialised file to the version from 1.0.
> Thus
> >>> the test demonstrates serialisation compatibility with the original
> >>> version. Adding 1.1 through 1.6 could be done although I do not see the
> >>> need. No fields were added until 1.7.
> >>>> WDYT?
> >>>>
> >>>>
> >>>>> Gary
> >>>>>
> >>>>>
> >>>>> On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <
> garydgregory@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
> >>>>>>
> >>>>>>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <
> alex.d.herbert@gmail.com
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> On 24/01/2020 13:34, Gary Gregory wrote:
> >>>>>>>>> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <
> >>> alex.d.herbert@gmail.com
> >>>>>>>>>> wrote:
> >>>>>>>>>>> On 23/01/2020 13:55, sebb wrote:
> >>>>>>>>>>>> I think we don't want temporary serialisation fixes to
> encourage
> >>>>>>> the
> >>>>>>>>>>>> use of serialisation.
> >>>>>>>>>>>>
> >>>>>>>>>>>> So I suggest that the Release Notes and Javadoc should point
> out
> >>>>>>> that
> >>>>>>>>>>>> although serialisation is possible, it is not fully supported,
> >>> and
> >>>>>>>>>>>> that there are plans to drop all serialisation support.
> >>>>>>>>>>> The javadoc for the new field that is not serialized have been
> >>>>>>>>>>> documented. This current code is able to deserialize a record
> >>>>>>> created
> >>>>>>>>>>> using version 1.0 and 1.6. I did not test the in between
> >>> releases.
> >>>>>>>>>>> Serialisation broke in 1.7.
> >>>>>>>>>>>
> >>>>>>>>>>> Should a note be added to the header for CSVRecord stating that
> >>> the
> >>>>>>>>>>> class is serialization compatible with version 1.0 - 1.6.
> Fields
> >>>>>>> added
> >>>>>>>>>>> after version 1.6 are not serialized and the intension is to
> >>> remove
> >>>>>>>>>>> serialisation support in version 2.0.
> >>>>>>>>>>>
> >>>>>>>>>>> WDYT?
> >>>>>>>>>> LGTM (apart from some spelling issues!)
> >>>>>>>>>>
> >>>>>>>>>> However, I think it's worth noting in the Release Notes as well.
> >>>>>>>>>>
> >>>>>>>>> I'm OK with what Sebb said.
> >>>>>>>>>
> >>>>>>>>> Gary
> >>>>>>>> OK. I'll update the description in the changes.xml for this
> release
> >>>>>>>> (which IIUC become the release notes)
> >>>>>>> This is not automatic, but yes, changes.xml can be used to generate
> >>> the RN
> >>>>>> I use the changes.xml file to generate the RNs.
> >>>>>>
> >>>>>> Gary
> >>>>>>
> >>>>>>
> >>>>>>>> and javadoc the CSVRecord in the
> >>>>>>>> class header.
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>> Alex
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>> Alex
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>>>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>> ---------------------------------------------------------------------
> >>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>>>
> >>>>>>>
> >>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
On 28/01/2020 15:15, Gary Gregory wrote:
> On Mon, Jan 27, 2020 at 7:58 PM Gary Gregory <ga...@gmail.com> wrote:
>
>> On Mon, Jan 27, 2020 at 6:54 PM Alex Herbert <al...@gmail.com>
>> wrote:
>>
>>> I’ve had a look at the Serialization of CSVRecord. Fields have been added
>>> and removed from CSVRecord as:
>>>
>>> 1.0
>>>
>>> /** The accumulated comments (if any) */
>>> private final String comment;
>>>
>>> /** The column name to index mapping. */
>>> private final Map<String, Integer> mapping;
>>>
>>> /** The record number. */
>>> private final long recordNumber;
>>>
>>> /** The values of the record */
>>> private final String[] values;
>>>
>>> 1.1
>>>
>>> // Added
>>>
>>> private final long characterPosition;
>>>
>>> 1.2 to 1.6
>>>
>>> No changes
>>>
>>> 1.7
>>>
>>> // Removed
>>>
>>> /** The column name to index mapping. */
>>> private final Map<String, Integer> mapping;
>>>
>>> // Added a non-serialisable field -> broken
>>>
>>> /** The parser that originates this record. */
>>> private final CSVParser parser;
>>>
>>> 1.8
>>>
>>> // Fixed by marking as transient
>>> private final transient CSVParser parser;
>>>
>>>
>>> If you write and read records in 1.8 the parser is not serialised. This
>>> contains the map between column names and indices. So the following methods
>>> are affected:
>>>
>>> boolean CSVRecord.isMapped(String);
>>> boolean CSVRecord.isSet(String);
>>> Map<String, String> CSVRecord.toMap();
>>> String CSVRecord.get(String);
>>>
>>> The original object would have valid returns for these. A deserialised
>>> version returns false for all map keys, the map representation is an empty
>>> map and get will throw as access by name is not supported.
>>>
>>>
>>> You can read records from 1.0 in using 1.8. It ignores the map that was
>>> serialised as part of the record. It also ignore the missing character
>>> position field and assigns it a default value of 0.
>>>
>>>
>>> Likewise you can read records from 1.8 in using 1.0. Here the map is
>>> missing and so all functionality linked to the map fails. The failures are
>>> exactly as for reading 1.0 in to 1.8.
>>>
>>>
>>> So this indicates you can serialise and deserialise between different
>>> versions back to 1.0 excluding 1.7. Deserialisation may not create records
>>> entirely. The following lists what should work with no loss of
>>> functionality in the destination version:
>>>
>>> Serialised from => deserialised to
>>>
>>> 1.0 => 1.0
>>> 1.1 - 1.6 => 1.0 (the character position field is ignored)
>>> 1.1 - 1.6 => 1.1 - 1.6
>>>
>>> The following will result in absent information:
>>>
>>> 1.0 => 1.1 - 1.6 (no character position)
>>> 1.0 => 1.8 (no character position, no mapping functionality)
>>> 1.1 - 1.6 => 1.8 (no mapping functionality)
>>> 1.8 => 1.0 - 1.8 (no mapping functionality)
>>>
>>> 1.7 is ignored as it cannot be serialised
>>>
>>>
>>> So even though you can deserialise between versions without an exception
>>> the functionality is impaired in certain cases, i.e. when jumping between
>>> 1.0-1.6 and 1.8. Since deserialisation can create an instance a change to
>>> the serial version ID is not warranted. The compatibility should be noted
>>> in the documentation, e.g. something like:
>>>
>>> * <p>
>>> * Note: Support for {@link Serializable} is scheduled to be removed in
>>> version 2.0.
>>> * In version 1.8 the mapping between the column header and the column
>>> index was
>>> * removed from the serialised state. The class maintains serialization
>>> compatibility
>>> * with versions pre-1.8 for the record values; these must be accessed by
>>> index
>>> * following deserialization. There will be loss of any functionally
>>> linked to the header
>>> * map when transferring serialised forms pre-1.8 to 1.8 and vice versa.
>>> * </p>
>>>
>>> Thoughts on this?
>>>
>> +1 to updating the docs.
>>
> Thanks for the update Alex. Do we want anything else for RC2?
Not from me. Serialization is fixed (for compatibility) and the docs now
state it won't be fully functional going forward. I hope that is the end
of that.
> Gary
>
>
>> Gary
>>
>>
>>> Alex
>>>
>>>> On 25 Jan 2020, at 23:02, Alex Herbert <al...@gmail.com>
>>> wrote:
>>>>
>>>>
>>>>> On 25 Jan 2020, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
>>>>>
>>>>> Hi Alex,
>>>>>
>>>>> Is there more you'd like to do for 1.8?
>>>>>
>>>> Not functionality. The remaining things are the documentation of:
>>>>
>>>> - the intent to remove Serialisation support to the CSVRecord in 2.0
>>>> - the intent to not support Serialisation of additional fields added
>>> after from 1.6
>>>> This is for the CSVRecord class header and to the release notes in
>>> changes.xml.
>>>> We have the test to show that you can deserialise from 1.6. I think
>>> that I should change the .bin serialised file to the version from 1.0. Thus
>>> the test demonstrates serialisation compatibility with the original
>>> version. Adding 1.1 through 1.6 could be done although I do not see the
>>> need. No fields were added until 1.7.
>>>> WDYT?
>>>>
>>>>
>>>>> Gary
>>>>>
>>>>>
>>>>> On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <ga...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
>>>>>>
>>>>>>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <alex.d.herbert@gmail.com
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 24/01/2020 13:34, Gary Gregory wrote:
>>>>>>>>> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <
>>> alex.d.herbert@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>>>> On 23/01/2020 13:55, sebb wrote:
>>>>>>>>>>>> I think we don't want temporary serialisation fixes to encourage
>>>>>>> the
>>>>>>>>>>>> use of serialisation.
>>>>>>>>>>>>
>>>>>>>>>>>> So I suggest that the Release Notes and Javadoc should point out
>>>>>>> that
>>>>>>>>>>>> although serialisation is possible, it is not fully supported,
>>> and
>>>>>>>>>>>> that there are plans to drop all serialisation support.
>>>>>>>>>>> The javadoc for the new field that is not serialized have been
>>>>>>>>>>> documented. This current code is able to deserialize a record
>>>>>>> created
>>>>>>>>>>> using version 1.0 and 1.6. I did not test the in between
>>> releases.
>>>>>>>>>>> Serialisation broke in 1.7.
>>>>>>>>>>>
>>>>>>>>>>> Should a note be added to the header for CSVRecord stating that
>>> the
>>>>>>>>>>> class is serialization compatible with version 1.0 - 1.6. Fields
>>>>>>> added
>>>>>>>>>>> after version 1.6 are not serialized and the intension is to
>>> remove
>>>>>>>>>>> serialisation support in version 2.0.
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>> LGTM (apart from some spelling issues!)
>>>>>>>>>>
>>>>>>>>>> However, I think it's worth noting in the Release Notes as well.
>>>>>>>>>>
>>>>>>>>> I'm OK with what Sebb said.
>>>>>>>>>
>>>>>>>>> Gary
>>>>>>>> OK. I'll update the description in the changes.xml for this release
>>>>>>>> (which IIUC become the release notes)
>>>>>>> This is not automatic, but yes, changes.xml can be used to generate
>>> the RN
>>>>>> I use the changes.xml file to generate the RNs.
>>>>>>
>>>>>> Gary
>>>>>>
>>>>>>
>>>>>>>> and javadoc the CSVRecord in the
>>>>>>>> class header.
>>>>>>> Thanks!
>>>>>>>
>>>>>>>> Alex
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Alex
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 27, 2020 at 7:58 PM Gary Gregory <ga...@gmail.com> wrote:
> On Mon, Jan 27, 2020 at 6:54 PM Alex Herbert <al...@gmail.com>
> wrote:
>
>> I’ve had a look at the Serialization of CSVRecord. Fields have been added
>> and removed from CSVRecord as:
>>
>> 1.0
>>
>> /** The accumulated comments (if any) */
>> private final String comment;
>>
>> /** The column name to index mapping. */
>> private final Map<String, Integer> mapping;
>>
>> /** The record number. */
>> private final long recordNumber;
>>
>> /** The values of the record */
>> private final String[] values;
>>
>> 1.1
>>
>> // Added
>>
>> private final long characterPosition;
>>
>> 1.2 to 1.6
>>
>> No changes
>>
>> 1.7
>>
>> // Removed
>>
>> /** The column name to index mapping. */
>> private final Map<String, Integer> mapping;
>>
>> // Added a non-serialisable field -> broken
>>
>> /** The parser that originates this record. */
>> private final CSVParser parser;
>>
>> 1.8
>>
>> // Fixed by marking as transient
>> private final transient CSVParser parser;
>>
>>
>> If you write and read records in 1.8 the parser is not serialised. This
>> contains the map between column names and indices. So the following methods
>> are affected:
>>
>> boolean CSVRecord.isMapped(String);
>> boolean CSVRecord.isSet(String);
>> Map<String, String> CSVRecord.toMap();
>> String CSVRecord.get(String);
>>
>> The original object would have valid returns for these. A deserialised
>> version returns false for all map keys, the map representation is an empty
>> map and get will throw as access by name is not supported.
>>
>>
>> You can read records from 1.0 in using 1.8. It ignores the map that was
>> serialised as part of the record. It also ignore the missing character
>> position field and assigns it a default value of 0.
>>
>>
>> Likewise you can read records from 1.8 in using 1.0. Here the map is
>> missing and so all functionality linked to the map fails. The failures are
>> exactly as for reading 1.0 in to 1.8.
>>
>>
>> So this indicates you can serialise and deserialise between different
>> versions back to 1.0 excluding 1.7. Deserialisation may not create records
>> entirely. The following lists what should work with no loss of
>> functionality in the destination version:
>>
>> Serialised from => deserialised to
>>
>> 1.0 => 1.0
>> 1.1 - 1.6 => 1.0 (the character position field is ignored)
>> 1.1 - 1.6 => 1.1 - 1.6
>>
>> The following will result in absent information:
>>
>> 1.0 => 1.1 - 1.6 (no character position)
>> 1.0 => 1.8 (no character position, no mapping functionality)
>> 1.1 - 1.6 => 1.8 (no mapping functionality)
>> 1.8 => 1.0 - 1.8 (no mapping functionality)
>>
>> 1.7 is ignored as it cannot be serialised
>>
>>
>> So even though you can deserialise between versions without an exception
>> the functionality is impaired in certain cases, i.e. when jumping between
>> 1.0-1.6 and 1.8. Since deserialisation can create an instance a change to
>> the serial version ID is not warranted. The compatibility should be noted
>> in the documentation, e.g. something like:
>>
>> * <p>
>> * Note: Support for {@link Serializable} is scheduled to be removed in
>> version 2.0.
>> * In version 1.8 the mapping between the column header and the column
>> index was
>> * removed from the serialised state. The class maintains serialization
>> compatibility
>> * with versions pre-1.8 for the record values; these must be accessed by
>> index
>> * following deserialization. There will be loss of any functionally
>> linked to the header
>> * map when transferring serialised forms pre-1.8 to 1.8 and vice versa.
>> * </p>
>>
>> Thoughts on this?
>>
>
> +1 to updating the docs.
>
Thanks for the update Alex. Do we want anything else for RC2?
Gary
>
> Gary
>
>
>>
>> Alex
>>
>> > On 25 Jan 2020, at 23:02, Alex Herbert <al...@gmail.com>
>> wrote:
>> >
>> >
>> >
>> >> On 25 Jan 2020, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
>> >>
>> >> Hi Alex,
>> >>
>> >> Is there more you'd like to do for 1.8?
>> >>
>> >
>> > Not functionality. The remaining things are the documentation of:
>> >
>> > - the intent to remove Serialisation support to the CSVRecord in 2.0
>> > - the intent to not support Serialisation of additional fields added
>> after from 1.6
>> >
>> > This is for the CSVRecord class header and to the release notes in
>> changes.xml.
>> >
>> > We have the test to show that you can deserialise from 1.6. I think
>> that I should change the .bin serialised file to the version from 1.0. Thus
>> the test demonstrates serialisation compatibility with the original
>> version. Adding 1.1 through 1.6 could be done although I do not see the
>> need. No fields were added until 1.7.
>> >
>> > WDYT?
>> >
>> >
>> >> Gary
>> >>
>> >>
>> >> On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <ga...@gmail.com>
>> >> wrote:
>> >>
>> >>> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
>> >>>
>> >>>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <alex.d.herbert@gmail.com
>> >
>> >>>> wrote:
>> >>>>>
>> >>>>>
>> >>>>> On 24/01/2020 13:34, Gary Gregory wrote:
>> >>>>>> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
>> >>>>>>
>> >>>>>>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <
>> alex.d.herbert@gmail.com
>> >>>>>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> On 23/01/2020 13:55, sebb wrote:
>> >>>>>>>>> I think we don't want temporary serialisation fixes to encourage
>> >>>> the
>> >>>>>>>>> use of serialisation.
>> >>>>>>>>>
>> >>>>>>>>> So I suggest that the Release Notes and Javadoc should point out
>> >>>> that
>> >>>>>>>>> although serialisation is possible, it is not fully supported,
>> and
>> >>>>>>>>> that there are plans to drop all serialisation support.
>> >>>>>>>> The javadoc for the new field that is not serialized have been
>> >>>>>>>> documented. This current code is able to deserialize a record
>> >>>> created
>> >>>>>>>> using version 1.0 and 1.6. I did not test the in between
>> releases.
>> >>>>>>>> Serialisation broke in 1.7.
>> >>>>>>>>
>> >>>>>>>> Should a note be added to the header for CSVRecord stating that
>> the
>> >>>>>>>> class is serialization compatible with version 1.0 - 1.6. Fields
>> >>>> added
>> >>>>>>>> after version 1.6 are not serialized and the intension is to
>> remove
>> >>>>>>>> serialisation support in version 2.0.
>> >>>>>>>>
>> >>>>>>>> WDYT?
>> >>>>>>> LGTM (apart from some spelling issues!)
>> >>>>>>>
>> >>>>>>> However, I think it's worth noting in the Release Notes as well.
>> >>>>>>>
>> >>>>>> I'm OK with what Sebb said.
>> >>>>>>
>> >>>>>> Gary
>> >>>>>
>> >>>>> OK. I'll update the description in the changes.xml for this release
>> >>>>> (which IIUC become the release notes)
>> >>>>
>> >>>> This is not automatic, but yes, changes.xml can be used to generate
>> the RN
>> >>>>
>> >>>
>> >>> I use the changes.xml file to generate the RNs.
>> >>>
>> >>> Gary
>> >>>
>> >>>
>> >>>>
>> >>>>> and javadoc the CSVRecord in the
>> >>>>> class header.
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>>> Alex
>> >>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>>> Alex
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>> ---------------------------------------------------------------------
>> >>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>> >>>>>>>>
>> >>>>>>>
>> ---------------------------------------------------------------------
>> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>>
>> ---------------------------------------------------------------------
>> >>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>>>> For additional commands, e-mail: dev-help@commons.apache.org
>> >>>>>
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> >>>> For additional commands, e-mail: dev-help@commons.apache.org
>> >>>>
>> >>>>
>> >
>>
>>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Jan 27, 2020 at 6:54 PM Alex Herbert <al...@gmail.com>
wrote:
> I’ve had a look at the Serialization of CSVRecord. Fields have been added
> and removed from CSVRecord as:
>
> 1.0
>
> /** The accumulated comments (if any) */
> private final String comment;
>
> /** The column name to index mapping. */
> private final Map<String, Integer> mapping;
>
> /** The record number. */
> private final long recordNumber;
>
> /** The values of the record */
> private final String[] values;
>
> 1.1
>
> // Added
>
> private final long characterPosition;
>
> 1.2 to 1.6
>
> No changes
>
> 1.7
>
> // Removed
>
> /** The column name to index mapping. */
> private final Map<String, Integer> mapping;
>
> // Added a non-serialisable field -> broken
>
> /** The parser that originates this record. */
> private final CSVParser parser;
>
> 1.8
>
> // Fixed by marking as transient
> private final transient CSVParser parser;
>
>
> If you write and read records in 1.8 the parser is not serialised. This
> contains the map between column names and indices. So the following methods
> are affected:
>
> boolean CSVRecord.isMapped(String);
> boolean CSVRecord.isSet(String);
> Map<String, String> CSVRecord.toMap();
> String CSVRecord.get(String);
>
> The original object would have valid returns for these. A deserialised
> version returns false for all map keys, the map representation is an empty
> map and get will throw as access by name is not supported.
>
>
> You can read records from 1.0 in using 1.8. It ignores the map that was
> serialised as part of the record. It also ignore the missing character
> position field and assigns it a default value of 0.
>
>
> Likewise you can read records from 1.8 in using 1.0. Here the map is
> missing and so all functionality linked to the map fails. The failures are
> exactly as for reading 1.0 in to 1.8.
>
>
> So this indicates you can serialise and deserialise between different
> versions back to 1.0 excluding 1.7. Deserialisation may not create records
> entirely. The following lists what should work with no loss of
> functionality in the destination version:
>
> Serialised from => deserialised to
>
> 1.0 => 1.0
> 1.1 - 1.6 => 1.0 (the character position field is ignored)
> 1.1 - 1.6 => 1.1 - 1.6
>
> The following will result in absent information:
>
> 1.0 => 1.1 - 1.6 (no character position)
> 1.0 => 1.8 (no character position, no mapping functionality)
> 1.1 - 1.6 => 1.8 (no mapping functionality)
> 1.8 => 1.0 - 1.8 (no mapping functionality)
>
> 1.7 is ignored as it cannot be serialised
>
>
> So even though you can deserialise between versions without an exception
> the functionality is impaired in certain cases, i.e. when jumping between
> 1.0-1.6 and 1.8. Since deserialisation can create an instance a change to
> the serial version ID is not warranted. The compatibility should be noted
> in the documentation, e.g. something like:
>
> * <p>
> * Note: Support for {@link Serializable} is scheduled to be removed in
> version 2.0.
> * In version 1.8 the mapping between the column header and the column
> index was
> * removed from the serialised state. The class maintains serialization
> compatibility
> * with versions pre-1.8 for the record values; these must be accessed by
> index
> * following deserialization. There will be loss of any functionally
> linked to the header
> * map when transferring serialised forms pre-1.8 to 1.8 and vice versa.
> * </p>
>
> Thoughts on this?
>
+1 to updating the docs.
Gary
>
> Alex
>
> > On 25 Jan 2020, at 23:02, Alex Herbert <al...@gmail.com> wrote:
> >
> >
> >
> >> On 25 Jan 2020, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
> >>
> >> Hi Alex,
> >>
> >> Is there more you'd like to do for 1.8?
> >>
> >
> > Not functionality. The remaining things are the documentation of:
> >
> > - the intent to remove Serialisation support to the CSVRecord in 2.0
> > - the intent to not support Serialisation of additional fields added
> after from 1.6
> >
> > This is for the CSVRecord class header and to the release notes in
> changes.xml.
> >
> > We have the test to show that you can deserialise from 1.6. I think that
> I should change the .bin serialised file to the version from 1.0. Thus the
> test demonstrates serialisation compatibility with the original version.
> Adding 1.1 through 1.6 could be done although I do not see the need. No
> fields were added until 1.7.
> >
> > WDYT?
> >
> >
> >> Gary
> >>
> >>
> >> On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <ga...@gmail.com>
> >> wrote:
> >>
> >>> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
> >>>
> >>>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <al...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>
> >>>>> On 24/01/2020 13:34, Gary Gregory wrote:
> >>>>>> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
> >>>>>>
> >>>>>>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <
> alex.d.herbert@gmail.com
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> On 23/01/2020 13:55, sebb wrote:
> >>>>>>>>> I think we don't want temporary serialisation fixes to encourage
> >>>> the
> >>>>>>>>> use of serialisation.
> >>>>>>>>>
> >>>>>>>>> So I suggest that the Release Notes and Javadoc should point out
> >>>> that
> >>>>>>>>> although serialisation is possible, it is not fully supported,
> and
> >>>>>>>>> that there are plans to drop all serialisation support.
> >>>>>>>> The javadoc for the new field that is not serialized have been
> >>>>>>>> documented. This current code is able to deserialize a record
> >>>> created
> >>>>>>>> using version 1.0 and 1.6. I did not test the in between releases.
> >>>>>>>> Serialisation broke in 1.7.
> >>>>>>>>
> >>>>>>>> Should a note be added to the header for CSVRecord stating that
> the
> >>>>>>>> class is serialization compatible with version 1.0 - 1.6. Fields
> >>>> added
> >>>>>>>> after version 1.6 are not serialized and the intension is to
> remove
> >>>>>>>> serialisation support in version 2.0.
> >>>>>>>>
> >>>>>>>> WDYT?
> >>>>>>> LGTM (apart from some spelling issues!)
> >>>>>>>
> >>>>>>> However, I think it's worth noting in the Release Notes as well.
> >>>>>>>
> >>>>>> I'm OK with what Sebb said.
> >>>>>>
> >>>>>> Gary
> >>>>>
> >>>>> OK. I'll update the description in the changes.xml for this release
> >>>>> (which IIUC become the release notes)
> >>>>
> >>>> This is not automatic, but yes, changes.xml can be used to generate
> the RN
> >>>>
> >>>
> >>> I use the changes.xml file to generate the RNs.
> >>>
> >>> Gary
> >>>
> >>>
> >>>>
> >>>>> and javadoc the CSVRecord in the
> >>>>> class header.
> >>>>
> >>>> Thanks!
> >>>>
> >>>>> Alex
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>>>> Alex
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>> ---------------------------------------------------------------------
> >>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>
> >>>>
> >
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
I’ve had a look at the Serialization of CSVRecord. Fields have been added and removed from CSVRecord as:
1.0
/** The accumulated comments (if any) */
private final String comment;
/** The column name to index mapping. */
private final Map<String, Integer> mapping;
/** The record number. */
private final long recordNumber;
/** The values of the record */
private final String[] values;
1.1
// Added
private final long characterPosition;
1.2 to 1.6
No changes
1.7
// Removed
/** The column name to index mapping. */
private final Map<String, Integer> mapping;
// Added a non-serialisable field -> broken
/** The parser that originates this record. */
private final CSVParser parser;
1.8
// Fixed by marking as transient
private final transient CSVParser parser;
If you write and read records in 1.8 the parser is not serialised. This contains the map between column names and indices. So the following methods are affected:
boolean CSVRecord.isMapped(String);
boolean CSVRecord.isSet(String);
Map<String, String> CSVRecord.toMap();
String CSVRecord.get(String);
The original object would have valid returns for these. A deserialised version returns false for all map keys, the map representation is an empty map and get will throw as access by name is not supported.
You can read records from 1.0 in using 1.8. It ignores the map that was serialised as part of the record. It also ignore the missing character position field and assigns it a default value of 0.
Likewise you can read records from 1.8 in using 1.0. Here the map is missing and so all functionality linked to the map fails. The failures are exactly as for reading 1.0 in to 1.8.
So this indicates you can serialise and deserialise between different versions back to 1.0 excluding 1.7. Deserialisation may not create records entirely. The following lists what should work with no loss of functionality in the destination version:
Serialised from => deserialised to
1.0 => 1.0
1.1 - 1.6 => 1.0 (the character position field is ignored)
1.1 - 1.6 => 1.1 - 1.6
The following will result in absent information:
1.0 => 1.1 - 1.6 (no character position)
1.0 => 1.8 (no character position, no mapping functionality)
1.1 - 1.6 => 1.8 (no mapping functionality)
1.8 => 1.0 - 1.8 (no mapping functionality)
1.7 is ignored as it cannot be serialised
So even though you can deserialise between versions without an exception the functionality is impaired in certain cases, i.e. when jumping between 1.0-1.6 and 1.8. Since deserialisation can create an instance a change to the serial version ID is not warranted. The compatibility should be noted in the documentation, e.g. something like:
* <p>
* Note: Support for {@link Serializable} is scheduled to be removed in version 2.0.
* In version 1.8 the mapping between the column header and the column index was
* removed from the serialised state. The class maintains serialization compatibility
* with versions pre-1.8 for the record values; these must be accessed by index
* following deserialization. There will be loss of any functionally linked to the header
* map when transferring serialised forms pre-1.8 to 1.8 and vice versa.
* </p>
Thoughts on this?
Alex
> On 25 Jan 2020, at 23:02, Alex Herbert <al...@gmail.com> wrote:
>
>
>
>> On 25 Jan 2020, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
>>
>> Hi Alex,
>>
>> Is there more you'd like to do for 1.8?
>>
>
> Not functionality. The remaining things are the documentation of:
>
> - the intent to remove Serialisation support to the CSVRecord in 2.0
> - the intent to not support Serialisation of additional fields added after from 1.6
>
> This is for the CSVRecord class header and to the release notes in changes.xml.
>
> We have the test to show that you can deserialise from 1.6. I think that I should change the .bin serialised file to the version from 1.0. Thus the test demonstrates serialisation compatibility with the original version. Adding 1.1 through 1.6 could be done although I do not see the need. No fields were added until 1.7.
>
> WDYT?
>
>
>> Gary
>>
>>
>> On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <ga...@gmail.com>
>> wrote:
>>
>>> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
>>>
>>>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <al...@gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> On 24/01/2020 13:34, Gary Gregory wrote:
>>>>>> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
>>>>>>
>>>>>>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <alex.d.herbert@gmail.com
>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 23/01/2020 13:55, sebb wrote:
>>>>>>>>> I think we don't want temporary serialisation fixes to encourage
>>>> the
>>>>>>>>> use of serialisation.
>>>>>>>>>
>>>>>>>>> So I suggest that the Release Notes and Javadoc should point out
>>>> that
>>>>>>>>> although serialisation is possible, it is not fully supported, and
>>>>>>>>> that there are plans to drop all serialisation support.
>>>>>>>> The javadoc for the new field that is not serialized have been
>>>>>>>> documented. This current code is able to deserialize a record
>>>> created
>>>>>>>> using version 1.0 and 1.6. I did not test the in between releases.
>>>>>>>> Serialisation broke in 1.7.
>>>>>>>>
>>>>>>>> Should a note be added to the header for CSVRecord stating that the
>>>>>>>> class is serialization compatible with version 1.0 - 1.6. Fields
>>>> added
>>>>>>>> after version 1.6 are not serialized and the intension is to remove
>>>>>>>> serialisation support in version 2.0.
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>> LGTM (apart from some spelling issues!)
>>>>>>>
>>>>>>> However, I think it's worth noting in the Release Notes as well.
>>>>>>>
>>>>>> I'm OK with what Sebb said.
>>>>>>
>>>>>> Gary
>>>>>
>>>>> OK. I'll update the description in the changes.xml for this release
>>>>> (which IIUC become the release notes)
>>>>
>>>> This is not automatic, but yes, changes.xml can be used to generate the RN
>>>>
>>>
>>> I use the changes.xml file to generate the RNs.
>>>
>>> Gary
>>>
>>>
>>>>
>>>>> and javadoc the CSVRecord in the
>>>>> class header.
>>>>
>>>> Thanks!
>>>>
>>>>> Alex
>>>>>
>>>>>>
>>>>>>
>>>>>>>> Alex
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
> On 25 Jan 2020, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
>
> Hi Alex,
>
> Is there more you'd like to do for 1.8?
>
Not functionality. The remaining things are the documentation of:
- the intent to remove Serialisation support to the CSVRecord in 2.0
- the intent to not support Serialisation of additional fields added after from 1.6
This is for the CSVRecord class header and to the release notes in changes.xml.
We have the test to show that you can deserialise from 1.6. I think that I should change the .bin serialised file to the version from 1.0. Thus the test demonstrates serialisation compatibility with the original version. Adding 1.1 through 1.6 could be done although I do not see the need. No fields were added until 1.7.
WDYT?
> Gary
>
>
> On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <ga...@gmail.com>
> wrote:
>
>> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
>>
>>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <al...@gmail.com>
>>> wrote:
>>>>
>>>>
>>>> On 24/01/2020 13:34, Gary Gregory wrote:
>>>>> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
>>>>>
>>>>>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <alex.d.herbert@gmail.com
>>>>
>>>>>> wrote:
>>>>>>>
>>>>>>> On 23/01/2020 13:55, sebb wrote:
>>>>>>>> I think we don't want temporary serialisation fixes to encourage
>>> the
>>>>>>>> use of serialisation.
>>>>>>>>
>>>>>>>> So I suggest that the Release Notes and Javadoc should point out
>>> that
>>>>>>>> although serialisation is possible, it is not fully supported, and
>>>>>>>> that there are plans to drop all serialisation support.
>>>>>>> The javadoc for the new field that is not serialized have been
>>>>>>> documented. This current code is able to deserialize a record
>>> created
>>>>>>> using version 1.0 and 1.6. I did not test the in between releases.
>>>>>>> Serialisation broke in 1.7.
>>>>>>>
>>>>>>> Should a note be added to the header for CSVRecord stating that the
>>>>>>> class is serialization compatible with version 1.0 - 1.6. Fields
>>> added
>>>>>>> after version 1.6 are not serialized and the intension is to remove
>>>>>>> serialisation support in version 2.0.
>>>>>>>
>>>>>>> WDYT?
>>>>>> LGTM (apart from some spelling issues!)
>>>>>>
>>>>>> However, I think it's worth noting in the Release Notes as well.
>>>>>>
>>>>> I'm OK with what Sebb said.
>>>>>
>>>>> Gary
>>>>
>>>> OK. I'll update the description in the changes.xml for this release
>>>> (which IIUC become the release notes)
>>>
>>> This is not automatic, but yes, changes.xml can be used to generate the RN
>>>
>>
>> I use the changes.xml file to generate the RNs.
>>
>> Gary
>>
>>
>>>
>>>> and javadoc the CSVRecord in the
>>>> class header.
>>>
>>> Thanks!
>>>
>>>> Alex
>>>>
>>>>>
>>>>>
>>>>>>> Alex
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
Hi Alex,
Is there more you'd like to do for 1.8?
Gary
On Fri, Jan 24, 2020 at 12:36 PM Gary Gregory <ga...@gmail.com>
wrote:
> On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
>
>> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <al...@gmail.com>
>> wrote:
>> >
>> >
>> > On 24/01/2020 13:34, Gary Gregory wrote:
>> > > On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
>> > >
>> > >> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <alex.d.herbert@gmail.com
>> >
>> > >> wrote:
>> > >>>
>> > >>> On 23/01/2020 13:55, sebb wrote:
>> > >>>> I think we don't want temporary serialisation fixes to encourage
>> the
>> > >>>> use of serialisation.
>> > >>>>
>> > >>>> So I suggest that the Release Notes and Javadoc should point out
>> that
>> > >>>> although serialisation is possible, it is not fully supported, and
>> > >>>> that there are plans to drop all serialisation support.
>> > >>> The javadoc for the new field that is not serialized have been
>> > >>> documented. This current code is able to deserialize a record
>> created
>> > >>> using version 1.0 and 1.6. I did not test the in between releases.
>> > >>> Serialisation broke in 1.7.
>> > >>>
>> > >>> Should a note be added to the header for CSVRecord stating that the
>> > >>> class is serialization compatible with version 1.0 - 1.6. Fields
>> added
>> > >>> after version 1.6 are not serialized and the intension is to remove
>> > >>> serialisation support in version 2.0.
>> > >>>
>> > >>> WDYT?
>> > >> LGTM (apart from some spelling issues!)
>> > >>
>> > >> However, I think it's worth noting in the Release Notes as well.
>> > >>
>> > > I'm OK with what Sebb said.
>> > >
>> > > Gary
>> >
>> > OK. I'll update the description in the changes.xml for this release
>> > (which IIUC become the release notes)
>>
>> This is not automatic, but yes, changes.xml can be used to generate the RN
>>
>
> I use the changes.xml file to generate the RNs.
>
> Gary
>
>
>>
>> > and javadoc the CSVRecord in the
>> > class header.
>>
>> Thanks!
>>
>> > Alex
>> >
>> > >
>> > >
>> > >>> Alex
>> > >>>
>> > >>>
>> > >>>
>> > >>>
>> ---------------------------------------------------------------------
>> > >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > >>> For additional commands, e-mail: dev-help@commons.apache.org
>> > >>>
>> > >> ---------------------------------------------------------------------
>> > >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > >> For additional commands, e-mail: dev-help@commons.apache.org
>> > >>
>> > >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Fri, Jan 24, 2020 at 11:09 AM sebb <se...@gmail.com> wrote:
> On Fri, 24 Jan 2020 at 15:05, Alex Herbert <al...@gmail.com>
> wrote:
> >
> >
> > On 24/01/2020 13:34, Gary Gregory wrote:
> > > On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
> > >
> > >> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <al...@gmail.com>
> > >> wrote:
> > >>>
> > >>> On 23/01/2020 13:55, sebb wrote:
> > >>>> I think we don't want temporary serialisation fixes to encourage the
> > >>>> use of serialisation.
> > >>>>
> > >>>> So I suggest that the Release Notes and Javadoc should point out
> that
> > >>>> although serialisation is possible, it is not fully supported, and
> > >>>> that there are plans to drop all serialisation support.
> > >>> The javadoc for the new field that is not serialized have been
> > >>> documented. This current code is able to deserialize a record created
> > >>> using version 1.0 and 1.6. I did not test the in between releases.
> > >>> Serialisation broke in 1.7.
> > >>>
> > >>> Should a note be added to the header for CSVRecord stating that the
> > >>> class is serialization compatible with version 1.0 - 1.6. Fields
> added
> > >>> after version 1.6 are not serialized and the intension is to remove
> > >>> serialisation support in version 2.0.
> > >>>
> > >>> WDYT?
> > >> LGTM (apart from some spelling issues!)
> > >>
> > >> However, I think it's worth noting in the Release Notes as well.
> > >>
> > > I'm OK with what Sebb said.
> > >
> > > Gary
> >
> > OK. I'll update the description in the changes.xml for this release
> > (which IIUC become the release notes)
>
> This is not automatic, but yes, changes.xml can be used to generate the RN
>
I use the changes.xml file to generate the RNs.
Gary
>
> > and javadoc the CSVRecord in the
> > class header.
>
> Thanks!
>
> > Alex
> >
> > >
> > >
> > >>> Alex
> > >>>
> > >>>
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >> For additional commands, e-mail: dev-help@commons.apache.org
> > >>
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by sebb <se...@gmail.com>.
On Fri, 24 Jan 2020 at 15:05, Alex Herbert <al...@gmail.com> wrote:
>
>
> On 24/01/2020 13:34, Gary Gregory wrote:
> > On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
> >
> >> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <al...@gmail.com>
> >> wrote:
> >>>
> >>> On 23/01/2020 13:55, sebb wrote:
> >>>> I think we don't want temporary serialisation fixes to encourage the
> >>>> use of serialisation.
> >>>>
> >>>> So I suggest that the Release Notes and Javadoc should point out that
> >>>> although serialisation is possible, it is not fully supported, and
> >>>> that there are plans to drop all serialisation support.
> >>> The javadoc for the new field that is not serialized have been
> >>> documented. This current code is able to deserialize a record created
> >>> using version 1.0 and 1.6. I did not test the in between releases.
> >>> Serialisation broke in 1.7.
> >>>
> >>> Should a note be added to the header for CSVRecord stating that the
> >>> class is serialization compatible with version 1.0 - 1.6. Fields added
> >>> after version 1.6 are not serialized and the intension is to remove
> >>> serialisation support in version 2.0.
> >>>
> >>> WDYT?
> >> LGTM (apart from some spelling issues!)
> >>
> >> However, I think it's worth noting in the Release Notes as well.
> >>
> > I'm OK with what Sebb said.
> >
> > Gary
>
> OK. I'll update the description in the changes.xml for this release
> (which IIUC become the release notes)
This is not automatic, but yes, changes.xml can be used to generate the RN
> and javadoc the CSVRecord in the
> class header.
Thanks!
> Alex
>
> >
> >
> >>> Alex
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
On 24/01/2020 13:34, Gary Gregory wrote:
> On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
>
>> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <al...@gmail.com>
>> wrote:
>>>
>>> On 23/01/2020 13:55, sebb wrote:
>>>> I think we don't want temporary serialisation fixes to encourage the
>>>> use of serialisation.
>>>>
>>>> So I suggest that the Release Notes and Javadoc should point out that
>>>> although serialisation is possible, it is not fully supported, and
>>>> that there are plans to drop all serialisation support.
>>> The javadoc for the new field that is not serialized have been
>>> documented. This current code is able to deserialize a record created
>>> using version 1.0 and 1.6. I did not test the in between releases.
>>> Serialisation broke in 1.7.
>>>
>>> Should a note be added to the header for CSVRecord stating that the
>>> class is serialization compatible with version 1.0 - 1.6. Fields added
>>> after version 1.6 are not serialized and the intension is to remove
>>> serialisation support in version 2.0.
>>>
>>> WDYT?
>> LGTM (apart from some spelling issues!)
>>
>> However, I think it's worth noting in the Release Notes as well.
>>
> I'm OK with what Sebb said.
>
> Gary
OK. I'll update the description in the changes.xml for this release
(which IIUC become the release notes) and javadoc the CSVRecord in the
class header.
Alex
>
>
>>> Alex
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Fri, Jan 24, 2020 at 6:14 AM sebb <se...@gmail.com> wrote:
> On Thu, 23 Jan 2020 at 18:10, Alex Herbert <al...@gmail.com>
> wrote:
> >
> >
> > On 23/01/2020 13:55, sebb wrote:
> > > I think we don't want temporary serialisation fixes to encourage the
> > > use of serialisation.
> > >
> > > So I suggest that the Release Notes and Javadoc should point out that
> > > although serialisation is possible, it is not fully supported, and
> > > that there are plans to drop all serialisation support.
> >
> > The javadoc for the new field that is not serialized have been
> > documented. This current code is able to deserialize a record created
> > using version 1.0 and 1.6. I did not test the in between releases.
> > Serialisation broke in 1.7.
> >
> > Should a note be added to the header for CSVRecord stating that the
> > class is serialization compatible with version 1.0 - 1.6. Fields added
> > after version 1.6 are not serialized and the intension is to remove
> > serialisation support in version 2.0.
> >
> > WDYT?
>
> LGTM (apart from some spelling issues!)
>
> However, I think it's worth noting in the Release Notes as well.
>
I'm OK with what Sebb said.
Gary
>
> > Alex
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by sebb <se...@gmail.com>.
On Thu, 23 Jan 2020 at 18:10, Alex Herbert <al...@gmail.com> wrote:
>
>
> On 23/01/2020 13:55, sebb wrote:
> > I think we don't want temporary serialisation fixes to encourage the
> > use of serialisation.
> >
> > So I suggest that the Release Notes and Javadoc should point out that
> > although serialisation is possible, it is not fully supported, and
> > that there are plans to drop all serialisation support.
>
> The javadoc for the new field that is not serialized have been
> documented. This current code is able to deserialize a record created
> using version 1.0 and 1.6. I did not test the in between releases.
> Serialisation broke in 1.7.
>
> Should a note be added to the header for CSVRecord stating that the
> class is serialization compatible with version 1.0 - 1.6. Fields added
> after version 1.6 are not serialized and the intension is to remove
> serialisation support in version 2.0.
>
> WDYT?
LGTM (apart from some spelling issues!)
However, I think it's worth noting in the Release Notes as well.
> Alex
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
On 23/01/2020 13:55, sebb wrote:
> I think we don't want temporary serialisation fixes to encourage the
> use of serialisation.
>
> So I suggest that the Release Notes and Javadoc should point out that
> although serialisation is possible, it is not fully supported, and
> that there are plans to drop all serialisation support.
The javadoc for the new field that is not serialized have been
documented. This current code is able to deserialize a record created
using version 1.0 and 1.6. I did not test the in between releases.
Serialisation broke in 1.7.
Should a note be added to the header for CSVRecord stating that the
class is serialization compatible with version 1.0 - 1.6. Fields added
after version 1.6 are not serialized and the intension is to remove
serialisation support in version 2.0.
WDYT?
Alex
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
[CSV] discuss 1.8 vote (was: Re: [VOTE] Release Apache Commons CSV
1.8 based on RC1)
Posted by Rob Tompkins <ch...@gmail.com>.
> On Jan 23, 2020, at 8:55 AM, sebb <se...@gmail.com> wrote:
>
> I think we don't want temporary serialisation fixes to encourage the
> use of serialisation.
>
> So I suggest that the Release Notes and Javadoc should point out that
> although serialisation is possible, it is not fully supported, and
> that there are plans to drop all serialisation support.
My 2c (maybe I’m not sufficiently contextualized here): I think that it’s ok to have breaking changes in a minor version release if it’s for a pointed security reason like removing serialization for the sake of security. In fact I think it’s almost a good idea to do that because it forces people to think about the security issues as opposed to just up versioning and assuming things will just work. That said, if we can make things more secure and have non-breaking changes, then that’s clearly ideal.
>
> On Tue, 21 Jan 2020 at 13:02, Alex Herbert <alex.d.herbert@gmail.com <ma...@gmail.com>> wrote:
>>
>>
>> On 20/01/2020 23:28, Gary Gregory wrote:
>>> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
>>> wrote:
>>>
>>>> Hi Gary,
>>>>
>>>> I raised a few niggles a while back with CSV and the discussion did not
>>>> receive a response on how to proceed.
>>>>
>>>> There is the major bug CSV-248 where the CSVRecord is not Serializable
>>>> [1]. This requires a decision on what to do to fix it. This bug is still
>>>> present in 1.8 RC1 as found by FindBugs [2].
>>>>
>>>> From what I can see the CSVRecord maintains a reference to the CSVParser.
>>>> This chain of objects maintained in memory is not serializable and leads
>>>> back to the original input Reader.
>>>>
>>>> I can see from the JApiCmp report that the serial version id was changed
>>>> for CSVRecord this release so there is still an intention to support
>>>> serialization. So this should be a blocker.
>>>>
>>>> I could not find a serialisation test in the unit tests for CSVRecord.
>>>> This quick test added to CSVRecordTest fails:
>>>>
>>>>
>>>> @Test
>>>> public void testSerialization() throws IOException {
>>>> CSVRecord shortRec;
>>>> try (final CSVParser parser = CSVParser.parse("a,b",
>>>> CSVFormat.newFormat(','))) {
>>>> shortRec = parser.iterator().next();
>>>> }
>>>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>>>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
>>>> oos.writeObject(shortRec);
>>>> }
>>>> }
>>>>
>>>> mvn test -Dtest=CSVRecordTest
>>>>
>>>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
>>>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
>>>> at
>>>> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>>>>
>>>> If I mark the field csvParser as transient it passes. So this is a problem
>>>> as raised by FindBugs.
>>>>
>>> Making the field transient would indeed fix this test but... some of the
>>> record APIs would then fail with NPEs... so we would be kicking the can
>>> down the road basically. Making the parser serializable is going in the
>>> wrong direction feature-wise IMO, so let's not go there. A serialization
>>> proxy would be less worse but should we provide such a thing? I would say
>>> no. I am OK with making the field transient despite the can kicking, so
>>> let's do that.
>>
>> I was adding some tests that the deserialised record behaves as if the
>> parser and header map are not available. I think this will be fine if we
>> update to this:
>>
>> private Map<String, Integer> getHeaderMapRaw() {
>> return parser == null ? null : parser.getHeaderMapRaw();
>> }
>>
>> Deserialisation from a form created with version 1.6 is fine. I added a
>> test for this. See current master.
>>
>> Alex
>>
>>
>>>
>>> Gary
>>>
>>>
>>>>
>>>> I also raised [3] the strange implementation of the CSVParser
>>>> getHeaderNames() which ignores null headers as they cannot be used as a key
>>>> into the map. However the list of column names could contain the null
>>>> values. This test currently fails:
>>>>
>>>> @Test
>>>> public void testHeaderNamesWithNull() throws IOException {
>>>> final Reader in = new
>>>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
>>>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>>>>
>>>> .withNullString("null")
>>>>
>>>> .withAllowMissingColumnNames()
>>>>
>>>> .parse(in).iterator();
>>>> final CSVRecord record = records.next();
>>>> assertEquals(Arrays.asList("header1", null, "header3"),
>>>> record.getParser().getHeaderNames());
>>>> }
>>>>
>>>> I am not saying it should pass but at least the documentation should state
>>>> the behaviour in this edge case. That is the list of header names may be
>>>> shorter than the number of columns when the parser is configured to allow
>>>> null headers. I’ve not raised a bug ticket for this as it is open to
>>>> opinion if this is by design or actually a bug. This issue is still present
>>>> in 1.8 RC1.
>>>>
>>>> Previously I suggested documentation changes for this and another edge
>>>> case using the header map to be added to the javadoc for getHeaderNames()
>>>> and getHeaderMap():
>>>>
>>>> - Documentation:
>>>>
>>>> The mapping is only guaranteed to be a one-to-one mapping if the record
>>>> was created with a format that does not allow duplicate or null header
>>>> names. Null headers are excluded from the map and duplicates can only map
>>>> to 1 column.
>>>>
>>>>
>>>> - Bug / Documentation
>>>>
>>>> The CSVParser only stores headers names in a list of header names if they
>>>> are not null. So the list can be shorter than the number of columns if you
>>>> use a format that allows empty headers and contains null column names.
>>>>
>>>>
>>>> The ultimate result is that we should document that the purpose of the
>>>> header names is to provide a list of non-null header names in the order
>>>> they occur in the header and thus represent keys that can be used in the
>>>> header map. In certain circumstances there may be more columns in the data
>>>> than there are header names.
>>>>
>>>>
>>>> Alex
>>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/CSV-248 <
>>>> https://issues.apache.org/jira/browse/CSV-248>
>>>>
>>>> [2]
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>>>> <
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>>>> [3] https://markmail.org/message/woti2iymecosihx6 <
>>>> https://markmail.org/message/woti2iymecosihx6>
>>>>
>>>>
>>>>
>>>>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>>>>>
>>>>> We have fixed quite a few bugs and added some significant enhancements
>>>>> since Apache Commons CSV 1.7 was released, so I would like to release
>>>>> Apache Commons CSV 1.8.
>>>>>
>>>>> Apache Commons CSV 1.8 RC1 is available for review here:
>>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
>>>>> revision 37670)
>>>>>
>>>>> The Git tag commons-csv-1.8-RC1 commit for this RC is
>>>>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>>>>>
>>>>>
>>>> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
>>>>> You may checkout this tag using:
>>>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>>>> --branch
>>>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>>>
>>>>> Maven artifacts are here:
>>>>>
>>>>>
>>>> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>>>>> These are the artifacts and their hashes:
>>>>>
>>>>> #Release SHA-512s
>>>>> #Sat Jan 18 12:01:01 EST 2020
>>>>>
>>>> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
>>>> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
>>>> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
>>>> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
>>>> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
>>>> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
>>>> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
>>>> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>>>>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
>>>>> japicmp clean package site deploy' using:
>>>>>
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\jdk1.8.0_241\jre
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> Additional tests with 'mvn -V clean test' using:
>>>>>
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\jdk1.8.0_241\jre
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\jdk-11.0.6
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\jdk-12.0.2
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\jdk-13.0.2
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\openjdk\jdk-14
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> JaCoCo fails on Java 15-EA because it does not know about class file
>>>> major
>>>>> version 59:
>>>>>
>>>>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
>>>> major
>>>>> version 59
>>>>> at
>>>>>
>>>> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>>>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
>>>>> Files\Java\openjdk\jdk-15
>>>>> Default locale: en_US, platform encoding: Cp1252
>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>>>
>>>>> Details of changes since 1.7 are in the release notes:
>>>>>
>>>>>
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>>>>>
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>>>>> Site:
>>>>>
>>>>>
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
>>>>> (note some *relative* links are broken and the 1.8 directories are not
>>>>> yet created - these will be OK once the site is deployed.)
>>>>>
>>>>> JApiCmp Report (compared to 1.7):
>>>>>
>>>>>
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>>>>> RAT Report:
>>>>>
>>>>>
>>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>>>>> KEYS:
>>>>> https://www.apache.org/dist/commons/KEYS
>>>>>
>>>>> Please review the release candidate and vote.
>>>>> This vote will close no sooner that 72 hours from now.
>>>>>
>>>>> [ ] +1 Release these artifacts
>>>>> [ ] +0 OK, but...
>>>>> [ ] -0 OK, but really should fix...
>>>>> [ ] -1 I oppose this release because...
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Gary Gregory,
>>>>> Release Manager (using key 86fdc7e2a11262cb)
>>>>>
>>>>> For following is intended as a helper and refresher for reviewers.
>>>>>
>>>>> Validating a release candidate
>>>>> ==============================
>>>>>
>>>>> These guidelines are NOT complete.
>>>>>
>>>>> Requirements: Git, Java, Maven.
>>>>>
>>>>> You can validate a release from a release candidate (RC) tag as follows.
>>>>>
>>>>> 1) Clone and checkout the RC tag
>>>>>
>>>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
>>>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>>> cd commons-csv-1.8-RC1
>>>>>
>>>>> 2) Check Apache licenses
>>>>>
>>>>> This step is not required if the site includes a RAT report page which
>>>> you
>>>>> then must check.
>>>>>
>>>>> mvn apache-rat:check
>>>>>
>>>>> 3) Check binary compatibility
>>>>>
>>>>> Older components still use Apache Clirr:
>>>>>
>>>>> This step is not required if the site includes a Clirr report page which
>>>>> you then must check.
>>>>>
>>>>> mvn clirr:check
>>>>>
>>>>> Newer components use JApiCmp with the japicmp Maven Profile:
>>>>>
>>>>> This step is not required if the site includes a JApiCmp report page
>>>> which
>>>>> you then must check.
>>>>>
>>>>> mvn install -DskipTests -P japicmp japicmp:cmp
>>>>>
>>>>> 4) Build the package
>>>>>
>>>>> mvn -V clean package
>>>>>
>>>>> You can record the Maven and Java version produced by -V in your VOTE
>>>> reply.
>>>>> To gather OS information from a command line:
>>>>> Windows: ver
>>>>> Linux: uname -a
>>>>>
>>>>> 5) Build the site for a single module project
>>>>>
>>>>> Note: Some plugins require the components to be installed instead of
>>>>> packaged.
>>>>>
>>>>> mvn site
>>>>> Check the site reports in:
>>>>> - Windows: target\site\index.html
>>>>> - Linux: target/site/index.html
>>>>>
>>>>> 6) Build the site for a multi-module project
>>>>>
>>>>> mvn site
>>>>> mvn site:stage
>>>>> Check the site reports in:
>>>>> - Windows: target\site\index.html
>>>>> - Linux: target/site/index.html
>>>>>
>>>>> -the end-
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org <ma...@commons.apache.org>
>> For additional commands, e-mail: dev-help@commons.apache.org <ma...@commons.apache.org>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org <ma...@commons.apache.org>
> For additional commands, e-mail: dev-help@commons.apache.org <ma...@commons.apache.org>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by sebb <se...@gmail.com>.
I think we don't want temporary serialisation fixes to encourage the
use of serialisation.
So I suggest that the Release Notes and Javadoc should point out that
although serialisation is possible, it is not fully supported, and
that there are plans to drop all serialisation support.
On Tue, 21 Jan 2020 at 13:02, Alex Herbert <al...@gmail.com> wrote:
>
>
> On 20/01/2020 23:28, Gary Gregory wrote:
> > On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
> > wrote:
> >
> >> Hi Gary,
> >>
> >> I raised a few niggles a while back with CSV and the discussion did not
> >> receive a response on how to proceed.
> >>
> >> There is the major bug CSV-248 where the CSVRecord is not Serializable
> >> [1]. This requires a decision on what to do to fix it. This bug is still
> >> present in 1.8 RC1 as found by FindBugs [2].
> >>
> >> From what I can see the CSVRecord maintains a reference to the CSVParser.
> >> This chain of objects maintained in memory is not serializable and leads
> >> back to the original input Reader.
> >>
> >> I can see from the JApiCmp report that the serial version id was changed
> >> for CSVRecord this release so there is still an intention to support
> >> serialization. So this should be a blocker.
> >>
> >> I could not find a serialisation test in the unit tests for CSVRecord.
> >> This quick test added to CSVRecordTest fails:
> >>
> >>
> >> @Test
> >> public void testSerialization() throws IOException {
> >> CSVRecord shortRec;
> >> try (final CSVParser parser = CSVParser.parse("a,b",
> >> CSVFormat.newFormat(','))) {
> >> shortRec = parser.iterator().next();
> >> }
> >> final ByteArrayOutputStream out = new ByteArrayOutputStream();
> >> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> >> oos.writeObject(shortRec);
> >> }
> >> }
> >>
> >> mvn test -Dtest=CSVRecordTest
> >>
> >> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> >> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> >> at
> >> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
> >>
> >> If I mark the field csvParser as transient it passes. So this is a problem
> >> as raised by FindBugs.
> >>
> > Making the field transient would indeed fix this test but... some of the
> > record APIs would then fail with NPEs... so we would be kicking the can
> > down the road basically. Making the parser serializable is going in the
> > wrong direction feature-wise IMO, so let's not go there. A serialization
> > proxy would be less worse but should we provide such a thing? I would say
> > no. I am OK with making the field transient despite the can kicking, so
> > let's do that.
>
> I was adding some tests that the deserialised record behaves as if the
> parser and header map are not available. I think this will be fine if we
> update to this:
>
> private Map<String, Integer> getHeaderMapRaw() {
> return parser == null ? null : parser.getHeaderMapRaw();
> }
>
> Deserialisation from a form created with version 1.6 is fine. I added a
> test for this. See current master.
>
> Alex
>
>
> >
> > Gary
> >
> >
> >>
> >> I also raised [3] the strange implementation of the CSVParser
> >> getHeaderNames() which ignores null headers as they cannot be used as a key
> >> into the map. However the list of column names could contain the null
> >> values. This test currently fails:
> >>
> >> @Test
> >> public void testHeaderNamesWithNull() throws IOException {
> >> final Reader in = new
> >> StringReader("header1,null,header3\n1,2,3\n4,5,6");
> >> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
> >>
> >> .withNullString("null")
> >>
> >> .withAllowMissingColumnNames()
> >>
> >> .parse(in).iterator();
> >> final CSVRecord record = records.next();
> >> assertEquals(Arrays.asList("header1", null, "header3"),
> >> record.getParser().getHeaderNames());
> >> }
> >>
> >> I am not saying it should pass but at least the documentation should state
> >> the behaviour in this edge case. That is the list of header names may be
> >> shorter than the number of columns when the parser is configured to allow
> >> null headers. I’ve not raised a bug ticket for this as it is open to
> >> opinion if this is by design or actually a bug. This issue is still present
> >> in 1.8 RC1.
> >>
> >> Previously I suggested documentation changes for this and another edge
> >> case using the header map to be added to the javadoc for getHeaderNames()
> >> and getHeaderMap():
> >>
> >> - Documentation:
> >>
> >> The mapping is only guaranteed to be a one-to-one mapping if the record
> >> was created with a format that does not allow duplicate or null header
> >> names. Null headers are excluded from the map and duplicates can only map
> >> to 1 column.
> >>
> >>
> >> - Bug / Documentation
> >>
> >> The CSVParser only stores headers names in a list of header names if they
> >> are not null. So the list can be shorter than the number of columns if you
> >> use a format that allows empty headers and contains null column names.
> >>
> >>
> >> The ultimate result is that we should document that the purpose of the
> >> header names is to provide a list of non-null header names in the order
> >> they occur in the header and thus represent keys that can be used in the
> >> header map. In certain circumstances there may be more columns in the data
> >> than there are header names.
> >>
> >>
> >> Alex
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/CSV-248 <
> >> https://issues.apache.org/jira/browse/CSV-248>
> >>
> >> [2]
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >> <
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >> [3] https://markmail.org/message/woti2iymecosihx6 <
> >> https://markmail.org/message/woti2iymecosihx6>
> >>
> >>
> >>
> >>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> >>>
> >>> We have fixed quite a few bugs and added some significant enhancements
> >>> since Apache Commons CSV 1.7 was released, so I would like to release
> >>> Apache Commons CSV 1.8.
> >>>
> >>> Apache Commons CSV 1.8 RC1 is available for review here:
> >>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> >>> revision 37670)
> >>>
> >>> The Git tag commons-csv-1.8-RC1 commit for this RC is
> >>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> >>>
> >>>
> >> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> >>> You may checkout this tag using:
> >>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> >> --branch
> >>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >>>
> >>> Maven artifacts are here:
> >>>
> >>>
> >> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> >>> These are the artifacts and their hashes:
> >>>
> >>> #Release SHA-512s
> >>> #Sat Jan 18 12:01:01 EST 2020
> >>>
> >> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> >> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> >> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> >> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> >> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> >> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> >> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> >> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> >>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> >>> japicmp clean package site deploy' using:
> >>>
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\jdk1.8.0_241\jre
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> Additional tests with 'mvn -V clean test' using:
> >>>
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\jdk1.8.0_241\jre
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\jdk-11.0.6
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\jdk-12.0.2
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\jdk-13.0.2
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\openjdk\jdk-14
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> JaCoCo fails on Java 15-EA because it does not know about class file
> >> major
> >>> version 59:
> >>>
> >>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
> >> major
> >>> version 59
> >>> at
> >>>
> >> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> >>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> >>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> >>> Files\Java\openjdk\jdk-15
> >>> Default locale: en_US, platform encoding: Cp1252
> >>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >>>
> >>> Details of changes since 1.7 are in the release notes:
> >>>
> >>>
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> >>>
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> >>> Site:
> >>>
> >>>
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> >>> (note some *relative* links are broken and the 1.8 directories are not
> >>> yet created - these will be OK once the site is deployed.)
> >>>
> >>> JApiCmp Report (compared to 1.7):
> >>>
> >>>
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> >>> RAT Report:
> >>>
> >>>
> >> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> >>> KEYS:
> >>> https://www.apache.org/dist/commons/KEYS
> >>>
> >>> Please review the release candidate and vote.
> >>> This vote will close no sooner that 72 hours from now.
> >>>
> >>> [ ] +1 Release these artifacts
> >>> [ ] +0 OK, but...
> >>> [ ] -0 OK, but really should fix...
> >>> [ ] -1 I oppose this release because...
> >>>
> >>> Thank you,
> >>>
> >>> Gary Gregory,
> >>> Release Manager (using key 86fdc7e2a11262cb)
> >>>
> >>> For following is intended as a helper and refresher for reviewers.
> >>>
> >>> Validating a release candidate
> >>> ==============================
> >>>
> >>> These guidelines are NOT complete.
> >>>
> >>> Requirements: Git, Java, Maven.
> >>>
> >>> You can validate a release from a release candidate (RC) tag as follows.
> >>>
> >>> 1) Clone and checkout the RC tag
> >>>
> >>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> >>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >>> cd commons-csv-1.8-RC1
> >>>
> >>> 2) Check Apache licenses
> >>>
> >>> This step is not required if the site includes a RAT report page which
> >> you
> >>> then must check.
> >>>
> >>> mvn apache-rat:check
> >>>
> >>> 3) Check binary compatibility
> >>>
> >>> Older components still use Apache Clirr:
> >>>
> >>> This step is not required if the site includes a Clirr report page which
> >>> you then must check.
> >>>
> >>> mvn clirr:check
> >>>
> >>> Newer components use JApiCmp with the japicmp Maven Profile:
> >>>
> >>> This step is not required if the site includes a JApiCmp report page
> >> which
> >>> you then must check.
> >>>
> >>> mvn install -DskipTests -P japicmp japicmp:cmp
> >>>
> >>> 4) Build the package
> >>>
> >>> mvn -V clean package
> >>>
> >>> You can record the Maven and Java version produced by -V in your VOTE
> >> reply.
> >>> To gather OS information from a command line:
> >>> Windows: ver
> >>> Linux: uname -a
> >>>
> >>> 5) Build the site for a single module project
> >>>
> >>> Note: Some plugins require the components to be installed instead of
> >>> packaged.
> >>>
> >>> mvn site
> >>> Check the site reports in:
> >>> - Windows: target\site\index.html
> >>> - Linux: target/site/index.html
> >>>
> >>> 6) Build the site for a multi-module project
> >>>
> >>> mvn site
> >>> mvn site:stage
> >>> Check the site reports in:
> >>> - Windows: target\site\index.html
> >>> - Linux: target/site/index.html
> >>>
> >>> -the end-
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
On 20/01/2020 23:28, Gary Gregory wrote:
> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
> wrote:
>
>> Hi Gary,
>>
>> I raised a few niggles a while back with CSV and the discussion did not
>> receive a response on how to proceed.
>>
>> There is the major bug CSV-248 where the CSVRecord is not Serializable
>> [1]. This requires a decision on what to do to fix it. This bug is still
>> present in 1.8 RC1 as found by FindBugs [2].
>>
>> From what I can see the CSVRecord maintains a reference to the CSVParser.
>> This chain of objects maintained in memory is not serializable and leads
>> back to the original input Reader.
>>
>> I can see from the JApiCmp report that the serial version id was changed
>> for CSVRecord this release so there is still an intention to support
>> serialization. So this should be a blocker.
>>
>> I could not find a serialisation test in the unit tests for CSVRecord.
>> This quick test added to CSVRecordTest fails:
>>
>>
>> @Test
>> public void testSerialization() throws IOException {
>> CSVRecord shortRec;
>> try (final CSVParser parser = CSVParser.parse("a,b",
>> CSVFormat.newFormat(','))) {
>> shortRec = parser.iterator().next();
>> }
>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
>> oos.writeObject(shortRec);
>> }
>> }
>>
>> mvn test -Dtest=CSVRecordTest
>>
>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
>> at
>> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>>
>> If I mark the field csvParser as transient it passes. So this is a problem
>> as raised by FindBugs.
>>
> Making the field transient would indeed fix this test but... some of the
> record APIs would then fail with NPEs... so we would be kicking the can
> down the road basically. Making the parser serializable is going in the
> wrong direction feature-wise IMO, so let's not go there. A serialization
> proxy would be less worse but should we provide such a thing? I would say
> no. I am OK with making the field transient despite the can kicking, so
> let's do that.
I was adding some tests that the deserialised record behaves as if the
parser and header map are not available. I think this will be fine if we
update to this:
private Map<String, Integer> getHeaderMapRaw() {
return parser == null ? null : parser.getHeaderMapRaw();
}
Deserialisation from a form created with version 1.6 is fine. I added a
test for this. See current master.
Alex
>
> Gary
>
>
>>
>> I also raised [3] the strange implementation of the CSVParser
>> getHeaderNames() which ignores null headers as they cannot be used as a key
>> into the map. However the list of column names could contain the null
>> values. This test currently fails:
>>
>> @Test
>> public void testHeaderNamesWithNull() throws IOException {
>> final Reader in = new
>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>>
>> .withNullString("null")
>>
>> .withAllowMissingColumnNames()
>>
>> .parse(in).iterator();
>> final CSVRecord record = records.next();
>> assertEquals(Arrays.asList("header1", null, "header3"),
>> record.getParser().getHeaderNames());
>> }
>>
>> I am not saying it should pass but at least the documentation should state
>> the behaviour in this edge case. That is the list of header names may be
>> shorter than the number of columns when the parser is configured to allow
>> null headers. I’ve not raised a bug ticket for this as it is open to
>> opinion if this is by design or actually a bug. This issue is still present
>> in 1.8 RC1.
>>
>> Previously I suggested documentation changes for this and another edge
>> case using the header map to be added to the javadoc for getHeaderNames()
>> and getHeaderMap():
>>
>> - Documentation:
>>
>> The mapping is only guaranteed to be a one-to-one mapping if the record
>> was created with a format that does not allow duplicate or null header
>> names. Null headers are excluded from the map and duplicates can only map
>> to 1 column.
>>
>>
>> - Bug / Documentation
>>
>> The CSVParser only stores headers names in a list of header names if they
>> are not null. So the list can be shorter than the number of columns if you
>> use a format that allows empty headers and contains null column names.
>>
>>
>> The ultimate result is that we should document that the purpose of the
>> header names is to provide a list of non-null header names in the order
>> they occur in the header and thus represent keys that can be used in the
>> header map. In certain circumstances there may be more columns in the data
>> than there are header names.
>>
>>
>> Alex
>>
>>
>> [1] https://issues.apache.org/jira/browse/CSV-248 <
>> https://issues.apache.org/jira/browse/CSV-248>
>>
>> [2]
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>> <
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
>> [3] https://markmail.org/message/woti2iymecosihx6 <
>> https://markmail.org/message/woti2iymecosihx6>
>>
>>
>>
>>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>>>
>>> We have fixed quite a few bugs and added some significant enhancements
>>> since Apache Commons CSV 1.7 was released, so I would like to release
>>> Apache Commons CSV 1.8.
>>>
>>> Apache Commons CSV 1.8 RC1 is available for review here:
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
>>> revision 37670)
>>>
>>> The Git tag commons-csv-1.8-RC1 commit for this RC is
>>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>>>
>>>
>> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
>>> You may checkout this tag using:
>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>> --branch
>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>
>>> Maven artifacts are here:
>>>
>>>
>> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>>> These are the artifacts and their hashes:
>>>
>>> #Release SHA-512s
>>> #Sat Jan 18 12:01:01 EST 2020
>>>
>> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
>> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
>> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
>> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
>> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
>> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
>> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
>> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
>>> japicmp clean package site deploy' using:
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk1.8.0_241\jre
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Additional tests with 'mvn -V clean test' using:
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk1.8.0_241\jre
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk-11.0.6
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk-12.0.2
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk-13.0.2
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\openjdk\jdk-14
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> JaCoCo fails on Java 15-EA because it does not know about class file
>> major
>>> version 59:
>>>
>>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
>> major
>>> version 59
>>> at
>>>
>> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\openjdk\jdk-15
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Details of changes since 1.7 are in the release notes:
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>>> Site:
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
>>> (note some *relative* links are broken and the 1.8 directories are not
>>> yet created - these will be OK once the site is deployed.)
>>>
>>> JApiCmp Report (compared to 1.7):
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>>> RAT Report:
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>>> KEYS:
>>> https://www.apache.org/dist/commons/KEYS
>>>
>>> Please review the release candidate and vote.
>>> This vote will close no sooner that 72 hours from now.
>>>
>>> [ ] +1 Release these artifacts
>>> [ ] +0 OK, but...
>>> [ ] -0 OK, but really should fix...
>>> [ ] -1 I oppose this release because...
>>>
>>> Thank you,
>>>
>>> Gary Gregory,
>>> Release Manager (using key 86fdc7e2a11262cb)
>>>
>>> For following is intended as a helper and refresher for reviewers.
>>>
>>> Validating a release candidate
>>> ==============================
>>>
>>> These guidelines are NOT complete.
>>>
>>> Requirements: Git, Java, Maven.
>>>
>>> You can validate a release from a release candidate (RC) tag as follows.
>>>
>>> 1) Clone and checkout the RC tag
>>>
>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>> cd commons-csv-1.8-RC1
>>>
>>> 2) Check Apache licenses
>>>
>>> This step is not required if the site includes a RAT report page which
>> you
>>> then must check.
>>>
>>> mvn apache-rat:check
>>>
>>> 3) Check binary compatibility
>>>
>>> Older components still use Apache Clirr:
>>>
>>> This step is not required if the site includes a Clirr report page which
>>> you then must check.
>>>
>>> mvn clirr:check
>>>
>>> Newer components use JApiCmp with the japicmp Maven Profile:
>>>
>>> This step is not required if the site includes a JApiCmp report page
>> which
>>> you then must check.
>>>
>>> mvn install -DskipTests -P japicmp japicmp:cmp
>>>
>>> 4) Build the package
>>>
>>> mvn -V clean package
>>>
>>> You can record the Maven and Java version produced by -V in your VOTE
>> reply.
>>> To gather OS information from a command line:
>>> Windows: ver
>>> Linux: uname -a
>>>
>>> 5) Build the site for a single module project
>>>
>>> Note: Some plugins require the components to be installed instead of
>>> packaged.
>>>
>>> mvn site
>>> Check the site reports in:
>>> - Windows: target\site\index.html
>>> - Linux: target/site/index.html
>>>
>>> 6) Build the site for a multi-module project
>>>
>>> mvn site
>>> mvn site:stage
>>> Check the site reports in:
>>> - Windows: target\site\index.html
>>> - Linux: target/site/index.html
>>>
>>> -the end-
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
> On 20 Jan 2020, at 23:28, Gary Gregory <ga...@gmail.com> wrote:
>
> On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <alex.d.herbert@gmail.com <ma...@gmail.com>>
> wrote:
>
>> Hi Gary,
>>
>> I raised a few niggles a while back with CSV and the discussion did not
>> receive a response on how to proceed.
>>
>> There is the major bug CSV-248 where the CSVRecord is not Serializable
>> [1]. This requires a decision on what to do to fix it. This bug is still
>> present in 1.8 RC1 as found by FindBugs [2].
>>
>> From what I can see the CSVRecord maintains a reference to the CSVParser.
>> This chain of objects maintained in memory is not serializable and leads
>> back to the original input Reader.
>>
>> I can see from the JApiCmp report that the serial version id was changed
>> for CSVRecord this release so there is still an intention to support
>> serialization. So this should be a blocker.
>>
>> I could not find a serialisation test in the unit tests for CSVRecord.
>> This quick test added to CSVRecordTest fails:
>>
>>
>> @Test
>> public void testSerialization() throws IOException {
>> CSVRecord shortRec;
>> try (final CSVParser parser = CSVParser.parse("a,b",
>> CSVFormat.newFormat(','))) {
>> shortRec = parser.iterator().next();
>> }
>> final ByteArrayOutputStream out = new ByteArrayOutputStream();
>> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
>> oos.writeObject(shortRec);
>> }
>> }
>>
>> mvn test -Dtest=CSVRecordTest
>>
>> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
>> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
>> at
>> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>>
>> If I mark the field csvParser as transient it passes. So this is a problem
>> as raised by FindBugs.
>>
>
> Making the field transient would indeed fix this test but... some of the
> record APIs would then fail with NPEs... so we would be kicking the can
> down the road basically. Making the parser serializable is going in the
> wrong direction feature-wise IMO, so let's not go there. A serialization
> proxy would be less worse but should we provide such a thing? I would say
> no. I am OK with making the field transient despite the can kicking, so
> let's do that.
This is all for a hypothetical problem at the moment. No one has reported being unable to (de)serialise when their app depends on this. So minimal effort to fix this makes sense.
Looking at the fix you pushed to master I think that the javadoc on getParser() should state that the parser is not part of the serialised state of the record.
>
> Gary
>
>
>>
>>
>> I also raised [3] the strange implementation of the CSVParser
>> getHeaderNames() which ignores null headers as they cannot be used as a key
>> into the map. However the list of column names could contain the null
>> values. This test currently fails:
>>
>> @Test
>> public void testHeaderNamesWithNull() throws IOException {
>> final Reader in = new
>> StringReader("header1,null,header3\n1,2,3\n4,5,6");
>> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>>
>> .withNullString("null")
>>
>> .withAllowMissingColumnNames()
>>
>> .parse(in).iterator();
>> final CSVRecord record = records.next();
>> assertEquals(Arrays.asList("header1", null, "header3"),
>> record.getParser().getHeaderNames());
>> }
>>
>> I am not saying it should pass but at least the documentation should state
>> the behaviour in this edge case. That is the list of header names may be
>> shorter than the number of columns when the parser is configured to allow
>> null headers. I’ve not raised a bug ticket for this as it is open to
>> opinion if this is by design or actually a bug. This issue is still present
>> in 1.8 RC1.
>>
>> Previously I suggested documentation changes for this and another edge
>> case using the header map to be added to the javadoc for getHeaderNames()
>> and getHeaderMap():
>>
>> - Documentation:
>>
>> The mapping is only guaranteed to be a one-to-one mapping if the record
>> was created with a format that does not allow duplicate or null header
>> names. Null headers are excluded from the map and duplicates can only map
>> to 1 column.
>>
>>
>> - Bug / Documentation
>>
>> The CSVParser only stores headers names in a list of header names if they
>> are not null. So the list can be shorter than the number of columns if you
>> use a format that allows empty headers and contains null column names.
>>
>>
>> The ultimate result is that we should document that the purpose of the
>> header names is to provide a list of non-null header names in the order
>> they occur in the header and thus represent keys that can be used in the
>> header map. In certain circumstances there may be more columns in the data
>> than there are header names.
>>
>>
>> Alex
>>
>>
>> [1] https://issues.apache.org/jira/browse/CSV-248 <
>> https://issues.apache.org/jira/browse/CSV-248 <https://issues.apache.org/jira/browse/CSV-248>>
>>
>> [2]
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html <https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html>
>> <
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html <https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html>
>>>
>>
>> [3] https://markmail.org/message/woti2iymecosihx6 <https://markmail.org/message/woti2iymecosihx6> <
>> https://markmail.org/message/woti2iymecosihx6 <https://markmail.org/message/woti2iymecosihx6>>
>>
>>
>>
>>> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>>>
>>> We have fixed quite a few bugs and added some significant enhancements
>>> since Apache Commons CSV 1.7 was released, so I would like to release
>>> Apache Commons CSV 1.8.
>>>
>>> Apache Commons CSV 1.8 RC1 is available for review here:
>>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
>>> revision 37670)
>>>
>>> The Git tag commons-csv-1.8-RC1 commit for this RC is
>>> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>>>
>>>
>> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
>>> You may checkout this tag using:
>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git
>> --branch
>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>>
>>> Maven artifacts are here:
>>>
>>>
>> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>>>
>>> These are the artifacts and their hashes:
>>>
>>> #Release SHA-512s
>>> #Sat Jan 18 12:01:01 EST 2020
>>>
>> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
>>>
>> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
>>>
>> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
>>>
>> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
>>>
>> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
>>>
>> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
>>>
>> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
>>>
>> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>>>
>>> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
>>> japicmp clean package site deploy' using:
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk1.8.0_241\jre
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Additional tests with 'mvn -V clean test' using:
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk1.8.0_241\jre
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk-11.0.6
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk-12.0.2
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\jdk-13.0.2
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\openjdk\jdk-14
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> JaCoCo fails on Java 15-EA because it does not know about class file
>> major
>>> version 59:
>>>
>>> Caused by: java.lang.IllegalArgumentException: Unsupported class file
>> major
>>> version 59
>>> at
>>>
>> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>>>
>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>>> Maven home: C:\Java\apache-maven-3.6.3\bin\..
>>> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
>>> Files\Java\openjdk\jdk-15
>>> Default locale: en_US, platform encoding: Cp1252
>>> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>>>
>>> Details of changes since 1.7 are in the release notes:
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>>>
>>> Site:
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
>>> (note some *relative* links are broken and the 1.8 directories are not
>>> yet created - these will be OK once the site is deployed.)
>>>
>>> JApiCmp Report (compared to 1.7):
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>>>
>>> RAT Report:
>>>
>>>
>> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>>>
>>> KEYS:
>>> https://www.apache.org/dist/commons/KEYS
>>>
>>> Please review the release candidate and vote.
>>> This vote will close no sooner that 72 hours from now.
>>>
>>> [ ] +1 Release these artifacts
>>> [ ] +0 OK, but...
>>> [ ] -0 OK, but really should fix...
>>> [ ] -1 I oppose this release because...
>>>
>>> Thank you,
>>>
>>> Gary Gregory,
>>> Release Manager (using key 86fdc7e2a11262cb)
>>>
>>> For following is intended as a helper and refresher for reviewers.
>>>
>>> Validating a release candidate
>>> ==============================
>>>
>>> These guidelines are NOT complete.
>>>
>>> Requirements: Git, Java, Maven.
>>>
>>> You can validate a release from a release candidate (RC) tag as follows.
>>>
>>> 1) Clone and checkout the RC tag
>>>
>>> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
>>> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>>> cd commons-csv-1.8-RC1
>>>
>>> 2) Check Apache licenses
>>>
>>> This step is not required if the site includes a RAT report page which
>> you
>>> then must check.
>>>
>>> mvn apache-rat:check
>>>
>>> 3) Check binary compatibility
>>>
>>> Older components still use Apache Clirr:
>>>
>>> This step is not required if the site includes a Clirr report page which
>>> you then must check.
>>>
>>> mvn clirr:check
>>>
>>> Newer components use JApiCmp with the japicmp Maven Profile:
>>>
>>> This step is not required if the site includes a JApiCmp report page
>> which
>>> you then must check.
>>>
>>> mvn install -DskipTests -P japicmp japicmp:cmp
>>>
>>> 4) Build the package
>>>
>>> mvn -V clean package
>>>
>>> You can record the Maven and Java version produced by -V in your VOTE
>> reply.
>>> To gather OS information from a command line:
>>> Windows: ver
>>> Linux: uname -a
>>>
>>> 5) Build the site for a single module project
>>>
>>> Note: Some plugins require the components to be installed instead of
>>> packaged.
>>>
>>> mvn site
>>> Check the site reports in:
>>> - Windows: target\site\index.html
>>> - Linux: target/site/index.html
>>>
>>> 6) Build the site for a multi-module project
>>>
>>> mvn site
>>> mvn site:stage
>>> Check the site reports in:
>>> - Windows: target\site\index.html
>>> - Linux: target/site/index.html
>>>
>>> -the end-
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
wrote:
> Hi Gary,
>
> I raised a few niggles a while back with CSV and the discussion did not
> receive a response on how to proceed.
>
> There is the major bug CSV-248 where the CSVRecord is not Serializable
> [1]. This requires a decision on what to do to fix it. This bug is still
> present in 1.8 RC1 as found by FindBugs [2].
>
> From what I can see the CSVRecord maintains a reference to the CSVParser.
> This chain of objects maintained in memory is not serializable and leads
> back to the original input Reader.
>
> I can see from the JApiCmp report that the serial version id was changed
> for CSVRecord this release so there is still an intention to support
> serialization. So this should be a blocker.
>
> I could not find a serialisation test in the unit tests for CSVRecord.
> This quick test added to CSVRecordTest fails:
>
>
> @Test
> public void testSerialization() throws IOException {
> CSVRecord shortRec;
> try (final CSVParser parser = CSVParser.parse("a,b",
> CSVFormat.newFormat(','))) {
> shortRec = parser.iterator().next();
> }
> final ByteArrayOutputStream out = new ByteArrayOutputStream();
> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> oos.writeObject(shortRec);
> }
> }
>
> mvn test -Dtest=CSVRecordTest
>
> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> at
> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>
> If I mark the field csvParser as transient it passes. So this is a problem
> as raised by FindBugs.
>
Making the field transient would indeed fix this test but... some of the
record APIs would then fail with NPEs... so we would be kicking the can
down the road basically. Making the parser serializable is going in the
wrong direction feature-wise IMO, so let's not go there. A serialization
proxy would be less worse but should we provide such a thing? I would say
no. I am OK with making the field transient despite the can kicking, so
let's do that.
Gary
>
>
> I also raised [3] the strange implementation of the CSVParser
> getHeaderNames() which ignores null headers as they cannot be used as a key
> into the map. However the list of column names could contain the null
> values. This test currently fails:
>
> @Test
> public void testHeaderNamesWithNull() throws IOException {
> final Reader in = new
> StringReader("header1,null,header3\n1,2,3\n4,5,6");
> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>
> .withNullString("null")
>
> .withAllowMissingColumnNames()
>
> .parse(in).iterator();
> final CSVRecord record = records.next();
> assertEquals(Arrays.asList("header1", null, "header3"),
> record.getParser().getHeaderNames());
> }
>
> I am not saying it should pass but at least the documentation should state
> the behaviour in this edge case. That is the list of header names may be
> shorter than the number of columns when the parser is configured to allow
> null headers. I’ve not raised a bug ticket for this as it is open to
> opinion if this is by design or actually a bug. This issue is still present
> in 1.8 RC1.
>
> Previously I suggested documentation changes for this and another edge
> case using the header map to be added to the javadoc for getHeaderNames()
> and getHeaderMap():
>
> - Documentation:
>
> The mapping is only guaranteed to be a one-to-one mapping if the record
> was created with a format that does not allow duplicate or null header
> names. Null headers are excluded from the map and duplicates can only map
> to 1 column.
>
>
> - Bug / Documentation
>
> The CSVParser only stores headers names in a list of header names if they
> are not null. So the list can be shorter than the number of columns if you
> use a format that allows empty headers and contains null column names.
>
>
> The ultimate result is that we should document that the purpose of the
> header names is to provide a list of non-null header names in the order
> they occur in the header and thus represent keys that can be used in the
> header map. In certain circumstances there may be more columns in the data
> than there are header names.
>
>
> Alex
>
>
> [1] https://issues.apache.org/jira/browse/CSV-248 <
> https://issues.apache.org/jira/browse/CSV-248>
>
> [2]
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> <
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >
>
> [3] https://markmail.org/message/woti2iymecosihx6 <
> https://markmail.org/message/woti2iymecosihx6>
>
>
>
> > On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> >
> > We have fixed quite a few bugs and added some significant enhancements
> > since Apache Commons CSV 1.7 was released, so I would like to release
> > Apache Commons CSV 1.8.
> >
> > Apache Commons CSV 1.8 RC1 is available for review here:
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> > revision 37670)
> >
> > The Git tag commons-csv-1.8-RC1 commit for this RC is
> > c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> >
> >
> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> > You may checkout this tag using:
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >
> > Maven artifacts are here:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> >
> > These are the artifacts and their hashes:
> >
> > #Release SHA-512s
> > #Sat Jan 18 12:01:01 EST 2020
> >
> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> >
> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> >
> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> >
> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> >
> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> >
> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> >
> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> >
> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> >
> > I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> > japicmp clean package site deploy' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Additional tests with 'mvn -V clean test' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-11.0.6
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-12.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-13.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-14
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > JaCoCo fails on Java 15-EA because it does not know about class file
> major
> > version 59:
> >
> > Caused by: java.lang.IllegalArgumentException: Unsupported class file
> major
> > version 59
> > at
> >
> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-15
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Details of changes since 1.7 are in the release notes:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> >
> > Site:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> > (note some *relative* links are broken and the 1.8 directories are not
> > yet created - these will be OK once the site is deployed.)
> >
> > JApiCmp Report (compared to 1.7):
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> >
> > RAT Report:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> >
> > KEYS:
> > https://www.apache.org/dist/commons/KEYS
> >
> > Please review the release candidate and vote.
> > This vote will close no sooner that 72 hours from now.
> >
> > [ ] +1 Release these artifacts
> > [ ] +0 OK, but...
> > [ ] -0 OK, but really should fix...
> > [ ] -1 I oppose this release because...
> >
> > Thank you,
> >
> > Gary Gregory,
> > Release Manager (using key 86fdc7e2a11262cb)
> >
> > For following is intended as a helper and refresher for reviewers.
> >
> > Validating a release candidate
> > ==============================
> >
> > These guidelines are NOT complete.
> >
> > Requirements: Git, Java, Maven.
> >
> > You can validate a release from a release candidate (RC) tag as follows.
> >
> > 1) Clone and checkout the RC tag
> >
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> > cd commons-csv-1.8-RC1
> >
> > 2) Check Apache licenses
> >
> > This step is not required if the site includes a RAT report page which
> you
> > then must check.
> >
> > mvn apache-rat:check
> >
> > 3) Check binary compatibility
> >
> > Older components still use Apache Clirr:
> >
> > This step is not required if the site includes a Clirr report page which
> > you then must check.
> >
> > mvn clirr:check
> >
> > Newer components use JApiCmp with the japicmp Maven Profile:
> >
> > This step is not required if the site includes a JApiCmp report page
> which
> > you then must check.
> >
> > mvn install -DskipTests -P japicmp japicmp:cmp
> >
> > 4) Build the package
> >
> > mvn -V clean package
> >
> > You can record the Maven and Java version produced by -V in your VOTE
> reply.
> > To gather OS information from a command line:
> > Windows: ver
> > Linux: uname -a
> >
> > 5) Build the site for a single module project
> >
> > Note: Some plugins require the components to be installed instead of
> > packaged.
> >
> > mvn site
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > 6) Build the site for a multi-module project
> >
> > mvn site
> > mvn site:stage
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > -the end-
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
wrote:
> Hi Gary,
>
> I raised a few niggles a while back with CSV and the discussion did not
> receive a response on how to proceed.
>
> There is the major bug CSV-248 where the CSVRecord is not Serializable
> [1]. This requires a decision on what to do to fix it. This bug is still
> present in 1.8 RC1 as found by FindBugs [2].
>
> From what I can see the CSVRecord maintains a reference to the CSVParser.
> This chain of objects maintained in memory is not serializable and leads
> back to the original input Reader.
>
> I can see from the JApiCmp report that the serial version id was changed
> for CSVRecord this release so there is still an intention to support
> serialization. So this should be a blocker.
>
> I could not find a serialisation test in the unit tests for CSVRecord.
> This quick test added to CSVRecordTest fails:
>
>
> @Test
> public void testSerialization() throws IOException {
> CSVRecord shortRec;
> try (final CSVParser parser = CSVParser.parse("a,b",
> CSVFormat.newFormat(','))) {
> shortRec = parser.iterator().next();
> }
> final ByteArrayOutputStream out = new ByteArrayOutputStream();
> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> oos.writeObject(shortRec);
> }
> }
>
> mvn test -Dtest=CSVRecordTest
>
> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> at
> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>
> If I mark the field csvParser as transient it passes. So this is a problem
> as raised by FindBugs.
>
>
>
> I also raised [3] the strange implementation of the CSVParser
> getHeaderNames() which ignores null headers as they cannot be used as a key
> into the map. However the list of column names could contain the null
> values. This test currently fails:
>
> @Test
> public void testHeaderNamesWithNull() throws IOException {
> final Reader in = new
> StringReader("header1,null,header3\n1,2,3\n4,5,6");
> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>
> .withNullString("null")
>
> .withAllowMissingColumnNames()
>
> .parse(in).iterator();
> final CSVRecord record = records.next();
> assertEquals(Arrays.asList("header1", null, "header3"),
> record.getParser().getHeaderNames());
> }
>
>
It seems obvious to me that withNullString applies to the header line like
any other but maybe this should be pointed out specifically in the Javadoc.
I am not saying it should pass but at least the documentation should state
> the behaviour in this edge case. That is the list of header names may be
> shorter than the number of columns when the parser is configured to allow
> null headers. I’ve not raised a bug ticket for this as it is open to
> opinion if this is by design or actually a bug. This issue is still present
> in 1.8 RC1.
design vs bug: If this falls into the bug bucket, how would you fix it? We
should do what is least surprising here IMO.
>
> Previously I suggested documentation changes for this and another edge
> case using the header map to be added to the javadoc for getHeaderNames()
> and getHeaderMap():
>
> - Documentation:
>
> The mapping is only guaranteed to be a one-to-one mapping if the record
> was created with a format that does not allow duplicate or null header
> names. Null headers are excluded from the map and duplicates can only map
> to 1 column.
>
I think I saw a feature request a while back to allow for duplicate column
names. That would obviously not work for querying a record, but you could
imagine that one should be able to write back out what was read in. I'm not
sure we have a test for this case.
>
> - Bug / Documentation
>
> The CSVParser only stores headers names in a list of header names if they
> are not null. So the list can be shorter than the number of columns if you
> use a format that allows empty headers and contains null column names.
>
>
> The ultimate result is that we should document that the purpose of the
> header names is to provide a list of non-null header names in the order
> they occur in the header and thus represent keys that can be used in the
> header map. In certain circumstances there may be more columns in the data
> than there are header names.
>
More docs is fine with me.
Gary
>
>
> Alex
>
>
> [1] https://issues.apache.org/jira/browse/CSV-248 <
> https://issues.apache.org/jira/browse/CSV-248>
>
> [2]
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> <
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >
>
> [3] https://markmail.org/message/woti2iymecosihx6 <
> https://markmail.org/message/woti2iymecosihx6>
>
>
>
> > On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> >
> > We have fixed quite a few bugs and added some significant enhancements
> > since Apache Commons CSV 1.7 was released, so I would like to release
> > Apache Commons CSV 1.8.
> >
> > Apache Commons CSV 1.8 RC1 is available for review here:
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> > revision 37670)
> >
> > The Git tag commons-csv-1.8-RC1 commit for this RC is
> > c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> >
> >
> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> > You may checkout this tag using:
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >
> > Maven artifacts are here:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> >
> > These are the artifacts and their hashes:
> >
> > #Release SHA-512s
> > #Sat Jan 18 12:01:01 EST 2020
> >
> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> >
> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> >
> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> >
> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> >
> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> >
> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> >
> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> >
> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> >
> > I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> > japicmp clean package site deploy' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Additional tests with 'mvn -V clean test' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-11.0.6
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-12.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-13.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-14
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > JaCoCo fails on Java 15-EA because it does not know about class file
> major
> > version 59:
> >
> > Caused by: java.lang.IllegalArgumentException: Unsupported class file
> major
> > version 59
> > at
> >
> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-15
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Details of changes since 1.7 are in the release notes:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> >
> > Site:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> > (note some *relative* links are broken and the 1.8 directories are not
> > yet created - these will be OK once the site is deployed.)
> >
> > JApiCmp Report (compared to 1.7):
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> >
> > RAT Report:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> >
> > KEYS:
> > https://www.apache.org/dist/commons/KEYS
> >
> > Please review the release candidate and vote.
> > This vote will close no sooner that 72 hours from now.
> >
> > [ ] +1 Release these artifacts
> > [ ] +0 OK, but...
> > [ ] -0 OK, but really should fix...
> > [ ] -1 I oppose this release because...
> >
> > Thank you,
> >
> > Gary Gregory,
> > Release Manager (using key 86fdc7e2a11262cb)
> >
> > For following is intended as a helper and refresher for reviewers.
> >
> > Validating a release candidate
> > ==============================
> >
> > These guidelines are NOT complete.
> >
> > Requirements: Git, Java, Maven.
> >
> > You can validate a release from a release candidate (RC) tag as follows.
> >
> > 1) Clone and checkout the RC tag
> >
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> > cd commons-csv-1.8-RC1
> >
> > 2) Check Apache licenses
> >
> > This step is not required if the site includes a RAT report page which
> you
> > then must check.
> >
> > mvn apache-rat:check
> >
> > 3) Check binary compatibility
> >
> > Older components still use Apache Clirr:
> >
> > This step is not required if the site includes a Clirr report page which
> > you then must check.
> >
> > mvn clirr:check
> >
> > Newer components use JApiCmp with the japicmp Maven Profile:
> >
> > This step is not required if the site includes a JApiCmp report page
> which
> > you then must check.
> >
> > mvn install -DskipTests -P japicmp japicmp:cmp
> >
> > 4) Build the package
> >
> > mvn -V clean package
> >
> > You can record the Maven and Java version produced by -V in your VOTE
> reply.
> > To gather OS information from a command line:
> > Windows: ver
> > Linux: uname -a
> >
> > 5) Build the site for a single module project
> >
> > Note: Some plugins require the components to be installed instead of
> > packaged.
> >
> > mvn site
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > 6) Build the site for a multi-module project
> >
> > mvn site
> > mvn site:stage
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > -the end-
>
>
[VOTE][CANCEL] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
I am canceling this RC so we can deal with these issues.
Gary
On Sun, Jan 19, 2020 at 7:39 AM Alex Herbert <al...@gmail.com>
wrote:
> Hi Gary,
>
> I raised a few niggles a while back with CSV and the discussion did not
> receive a response on how to proceed.
>
> There is the major bug CSV-248 where the CSVRecord is not Serializable
> [1]. This requires a decision on what to do to fix it. This bug is still
> present in 1.8 RC1 as found by FindBugs [2].
>
> From what I can see the CSVRecord maintains a reference to the CSVParser.
> This chain of objects maintained in memory is not serializable and leads
> back to the original input Reader.
>
> I can see from the JApiCmp report that the serial version id was changed
> for CSVRecord this release so there is still an intention to support
> serialization. So this should be a blocker.
>
> I could not find a serialisation test in the unit tests for CSVRecord.
> This quick test added to CSVRecordTest fails:
>
>
> @Test
> public void testSerialization() throws IOException {
> CSVRecord shortRec;
> try (final CSVParser parser = CSVParser.parse("a,b",
> CSVFormat.newFormat(','))) {
> shortRec = parser.iterator().next();
> }
> final ByteArrayOutputStream out = new ByteArrayOutputStream();
> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> oos.writeObject(shortRec);
> }
> }
>
> mvn test -Dtest=CSVRecordTest
>
> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> at
> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>
> If I mark the field csvParser as transient it passes. So this is a problem
> as raised by FindBugs.
>
>
>
> I also raised [3] the strange implementation of the CSVParser
> getHeaderNames() which ignores null headers as they cannot be used as a key
> into the map. However the list of column names could contain the null
> values. This test currently fails:
>
> @Test
> public void testHeaderNamesWithNull() throws IOException {
> final Reader in = new
> StringReader("header1,null,header3\n1,2,3\n4,5,6");
> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
>
> .withNullString("null")
>
> .withAllowMissingColumnNames()
>
> .parse(in).iterator();
> final CSVRecord record = records.next();
> assertEquals(Arrays.asList("header1", null, "header3"),
> record.getParser().getHeaderNames());
> }
>
> I am not saying it should pass but at least the documentation should state
> the behaviour in this edge case. That is the list of header names may be
> shorter than the number of columns when the parser is configured to allow
> null headers. I’ve not raised a bug ticket for this as it is open to
> opinion if this is by design or actually a bug. This issue is still present
> in 1.8 RC1.
>
> Previously I suggested documentation changes for this and another edge
> case using the header map to be added to the javadoc for getHeaderNames()
> and getHeaderMap():
>
> - Documentation:
>
> The mapping is only guaranteed to be a one-to-one mapping if the record
> was created with a format that does not allow duplicate or null header
> names. Null headers are excluded from the map and duplicates can only map
> to 1 column.
>
>
> - Bug / Documentation
>
> The CSVParser only stores headers names in a list of header names if they
> are not null. So the list can be shorter than the number of columns if you
> use a format that allows empty headers and contains null column names.
>
>
> The ultimate result is that we should document that the purpose of the
> header names is to provide a list of non-null header names in the order
> they occur in the header and thus represent keys that can be used in the
> header map. In certain circumstances there may be more columns in the data
> than there are header names.
>
>
> Alex
>
>
> [1] https://issues.apache.org/jira/browse/CSV-248 <
> https://issues.apache.org/jira/browse/CSV-248>
>
> [2]
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> <
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >
>
> [3] https://markmail.org/message/woti2iymecosihx6 <
> https://markmail.org/message/woti2iymecosihx6>
>
>
>
> > On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> >
> > We have fixed quite a few bugs and added some significant enhancements
> > since Apache Commons CSV 1.7 was released, so I would like to release
> > Apache Commons CSV 1.8.
> >
> > Apache Commons CSV 1.8 RC1 is available for review here:
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> > revision 37670)
> >
> > The Git tag commons-csv-1.8-RC1 commit for this RC is
> > c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> >
> >
> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> > You may checkout this tag using:
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >
> > Maven artifacts are here:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> >
> > These are the artifacts and their hashes:
> >
> > #Release SHA-512s
> > #Sat Jan 18 12:01:01 EST 2020
> >
> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> >
> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> >
> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> >
> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> >
> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> >
> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> >
> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> >
> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> >
> > I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> > japicmp clean package site deploy' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Additional tests with 'mvn -V clean test' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-11.0.6
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-12.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-13.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-14
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > JaCoCo fails on Java 15-EA because it does not know about class file
> major
> > version 59:
> >
> > Caused by: java.lang.IllegalArgumentException: Unsupported class file
> major
> > version 59
> > at
> >
> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-15
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Details of changes since 1.7 are in the release notes:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> >
> > Site:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> > (note some *relative* links are broken and the 1.8 directories are not
> > yet created - these will be OK once the site is deployed.)
> >
> > JApiCmp Report (compared to 1.7):
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> >
> > RAT Report:
> >
> >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> >
> > KEYS:
> > https://www.apache.org/dist/commons/KEYS
> >
> > Please review the release candidate and vote.
> > This vote will close no sooner that 72 hours from now.
> >
> > [ ] +1 Release these artifacts
> > [ ] +0 OK, but...
> > [ ] -0 OK, but really should fix...
> > [ ] -1 I oppose this release because...
> >
> > Thank you,
> >
> > Gary Gregory,
> > Release Manager (using key 86fdc7e2a11262cb)
> >
> > For following is intended as a helper and refresher for reviewers.
> >
> > Validating a release candidate
> > ==============================
> >
> > These guidelines are NOT complete.
> >
> > Requirements: Git, Java, Maven.
> >
> > You can validate a release from a release candidate (RC) tag as follows.
> >
> > 1) Clone and checkout the RC tag
> >
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> > cd commons-csv-1.8-RC1
> >
> > 2) Check Apache licenses
> >
> > This step is not required if the site includes a RAT report page which
> you
> > then must check.
> >
> > mvn apache-rat:check
> >
> > 3) Check binary compatibility
> >
> > Older components still use Apache Clirr:
> >
> > This step is not required if the site includes a Clirr report page which
> > you then must check.
> >
> > mvn clirr:check
> >
> > Newer components use JApiCmp with the japicmp Maven Profile:
> >
> > This step is not required if the site includes a JApiCmp report page
> which
> > you then must check.
> >
> > mvn install -DskipTests -P japicmp japicmp:cmp
> >
> > 4) Build the package
> >
> > mvn -V clean package
> >
> > You can record the Maven and Java version produced by -V in your VOTE
> reply.
> > To gather OS information from a command line:
> > Windows: ver
> > Linux: uname -a
> >
> > 5) Build the site for a single module project
> >
> > Note: Some plugins require the components to be installed instead of
> > packaged.
> >
> > mvn site
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > 6) Build the site for a multi-module project
> >
> > mvn site
> > mvn site:stage
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > -the end-
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
> On 20 Jan 2020, at 00:54, sebb <se...@gmail.com> wrote:
>
> What is the use case for needing serialisation?
> It's a lot of effort to maintain a serialisable class, and it opens
> the class to deserialisation attacks.
I don’t have a use case. But the class used to support serialization back to the code tagged as CSV_1.0. Putting out new releases that do not support it is breaking binary compatibility.
1.7 was the first to break compatibility. The live site reports it as such [1].
I will state that I voted +1 on release 1.7. Somehow the issue was missed then and it has bugged me ever since.
[1] https://commons.apache.org/proper/commons-csv/findbugs.html <https://commons.apache.org/proper/commons-csv/findbugs.html>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Gary Gregory <ga...@gmail.com>.
On Sun, Jan 19, 2020 at 7:55 PM sebb <se...@gmail.com> wrote:
> What is the use case for needing serialisation?
> It's a lot of effort to maintain a serialisable class, and it opens
> the class to deserialisation attacks.
>
I think the larger context is whether we can effectively remove (or leave
broken) serialization outside of a major version.
Gary
>
> On Sun, 19 Jan 2020 at 12:39, Alex Herbert <al...@gmail.com>
> wrote:
> >
> > Hi Gary,
> >
> > I raised a few niggles a while back with CSV and the discussion did not
> receive a response on how to proceed.
> >
> > There is the major bug CSV-248 where the CSVRecord is not Serializable
> [1]. This requires a decision on what to do to fix it. This bug is still
> present in 1.8 RC1 as found by FindBugs [2].
> >
> > From what I can see the CSVRecord maintains a reference to the
> CSVParser. This chain of objects maintained in memory is not serializable
> and leads back to the original input Reader.
> >
> > I can see from the JApiCmp report that the serial version id was changed
> for CSVRecord this release so there is still an intention to support
> serialization. So this should be a blocker.
> >
> > I could not find a serialisation test in the unit tests for CSVRecord.
> This quick test added to CSVRecordTest fails:
> >
> >
> > @Test
> > public void testSerialization() throws IOException {
> > CSVRecord shortRec;
> > try (final CSVParser parser = CSVParser.parse("a,b",
> CSVFormat.newFormat(','))) {
> > shortRec = parser.iterator().next();
> > }
> > final ByteArrayOutputStream out = new ByteArrayOutputStream();
> > try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> > oos.writeObject(shortRec);
> > }
> > }
> >
> > mvn test -Dtest=CSVRecordTest
> >
> > [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> > java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> > at
> org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
> >
> > If I mark the field csvParser as transient it passes. So this is a
> problem as raised by FindBugs.
> >
> >
> >
> > I also raised [3] the strange implementation of the CSVParser
> getHeaderNames() which ignores null headers as they cannot be used as a key
> into the map. However the list of column names could contain the null
> values. This test currently fails:
> >
> > @Test
> > public void testHeaderNamesWithNull() throws IOException {
> > final Reader in = new
> StringReader("header1,null,header3\n1,2,3\n4,5,6");
> > final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
> >
> .withNullString("null")
> >
> .withAllowMissingColumnNames()
> >
> .parse(in).iterator();
> > final CSVRecord record = records.next();
> > assertEquals(Arrays.asList("header1", null, "header3"),
> record.getParser().getHeaderNames());
> > }
> >
> > I am not saying it should pass but at least the documentation should
> state the behaviour in this edge case. That is the list of header names may
> be shorter than the number of columns when the parser is configured to
> allow null headers. I’ve not raised a bug ticket for this as it is open to
> opinion if this is by design or actually a bug. This issue is still present
> in 1.8 RC1.
> >
> > Previously I suggested documentation changes for this and another edge
> case using the header map to be added to the javadoc for getHeaderNames()
> and getHeaderMap():
> >
> > - Documentation:
> >
> > The mapping is only guaranteed to be a one-to-one mapping if the record
> was created with a format that does not allow duplicate or null header
> names. Null headers are excluded from the map and duplicates can only map
> to 1 column.
> >
> >
> > - Bug / Documentation
> >
> > The CSVParser only stores headers names in a list of header names if
> they are not null. So the list can be shorter than the number of columns if
> you use a format that allows empty headers and contains null column names.
> >
> >
> > The ultimate result is that we should document that the purpose of the
> header names is to provide a list of non-null header names in the order
> they occur in the header and thus represent keys that can be used in the
> header map. In certain circumstances there may be more columns in the data
> than there are header names.
> >
> >
> > Alex
> >
> >
> > [1] https://issues.apache.org/jira/browse/CSV-248 <
> https://issues.apache.org/jira/browse/CSV-248>
> >
> > [2]
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> <
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html
> >
> >
> > [3] https://markmail.org/message/woti2iymecosihx6 <
> https://markmail.org/message/woti2iymecosihx6>
> >
> >
> >
> > > On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> > >
> > > We have fixed quite a few bugs and added some significant enhancements
> > > since Apache Commons CSV 1.7 was released, so I would like to release
> > > Apache Commons CSV 1.8.
> > >
> > > Apache Commons CSV 1.8 RC1 is available for review here:
> > > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> > > revision 37670)
> > >
> > > The Git tag commons-csv-1.8-RC1 commit for this RC is
> > > c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> > >
> > >
> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> > > You may checkout this tag using:
> > > git clone https://gitbox.apache.org/repos/asf/commons-csv.git
> --branch
> > > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> > >
> > > Maven artifacts are here:
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> > >
> > > These are the artifacts and their hashes:
> > >
> > > #Release SHA-512s
> > > #Sat Jan 18 12:01:01 EST 2020
> > >
> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> > >
> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> > >
> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> > >
> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> > >
> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> > >
> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> > >
> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> > >
> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> > >
> > > I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> > > japicmp clean package site deploy' using:
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime:
> C:\Program
> > > Files\Java\jdk1.8.0_241\jre
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > Additional tests with 'mvn -V clean test' using:
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime:
> C:\Program
> > > Files\Java\jdk1.8.0_241\jre
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> > > Files\Java\jdk-11.0.6
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > > Files\Java\jdk-12.0.2
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > > Files\Java\jdk-13.0.2
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> > > Files\Java\openjdk\jdk-14
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > JaCoCo fails on Java 15-EA because it does not know about class file
> major
> > > version 59:
> > >
> > > Caused by: java.lang.IllegalArgumentException: Unsupported class file
> major
> > > version 59
> > > at
> > >
> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > > Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> > > Files\Java\openjdk\jdk-15
> > > Default locale: en_US, platform encoding: Cp1252
> > > OS name: "windows 10", version: "10.0", arch: "amd64", family:
> "windows"
> > >
> > > Details of changes since 1.7 are in the release notes:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> > >
> > >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> > >
> > > Site:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> > > (note some *relative* links are broken and the 1.8 directories are
> not
> > > yet created - these will be OK once the site is deployed.)
> > >
> > > JApiCmp Report (compared to 1.7):
> > >
> > >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> > >
> > > RAT Report:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> > >
> > > KEYS:
> > > https://www.apache.org/dist/commons/KEYS
> > >
> > > Please review the release candidate and vote.
> > > This vote will close no sooner that 72 hours from now.
> > >
> > > [ ] +1 Release these artifacts
> > > [ ] +0 OK, but...
> > > [ ] -0 OK, but really should fix...
> > > [ ] -1 I oppose this release because...
> > >
> > > Thank you,
> > >
> > > Gary Gregory,
> > > Release Manager (using key 86fdc7e2a11262cb)
> > >
> > > For following is intended as a helper and refresher for reviewers.
> > >
> > > Validating a release candidate
> > > ==============================
> > >
> > > These guidelines are NOT complete.
> > >
> > > Requirements: Git, Java, Maven.
> > >
> > > You can validate a release from a release candidate (RC) tag as
> follows.
> > >
> > > 1) Clone and checkout the RC tag
> > >
> > > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> > > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> > > cd commons-csv-1.8-RC1
> > >
> > > 2) Check Apache licenses
> > >
> > > This step is not required if the site includes a RAT report page which
> you
> > > then must check.
> > >
> > > mvn apache-rat:check
> > >
> > > 3) Check binary compatibility
> > >
> > > Older components still use Apache Clirr:
> > >
> > > This step is not required if the site includes a Clirr report page
> which
> > > you then must check.
> > >
> > > mvn clirr:check
> > >
> > > Newer components use JApiCmp with the japicmp Maven Profile:
> > >
> > > This step is not required if the site includes a JApiCmp report page
> which
> > > you then must check.
> > >
> > > mvn install -DskipTests -P japicmp japicmp:cmp
> > >
> > > 4) Build the package
> > >
> > > mvn -V clean package
> > >
> > > You can record the Maven and Java version produced by -V in your VOTE
> reply.
> > > To gather OS information from a command line:
> > > Windows: ver
> > > Linux: uname -a
> > >
> > > 5) Build the site for a single module project
> > >
> > > Note: Some plugins require the components to be installed instead of
> > > packaged.
> > >
> > > mvn site
> > > Check the site reports in:
> > > - Windows: target\site\index.html
> > > - Linux: target/site/index.html
> > >
> > > 6) Build the site for a multi-module project
> > >
> > > mvn site
> > > mvn site:stage
> > > Check the site reports in:
> > > - Windows: target\site\index.html
> > > - Linux: target/site/index.html
> > >
> > > -the end-
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by sebb <se...@gmail.com>.
What is the use case for needing serialisation?
It's a lot of effort to maintain a serialisable class, and it opens
the class to deserialisation attacks.
On Sun, 19 Jan 2020 at 12:39, Alex Herbert <al...@gmail.com> wrote:
>
> Hi Gary,
>
> I raised a few niggles a while back with CSV and the discussion did not receive a response on how to proceed.
>
> There is the major bug CSV-248 where the CSVRecord is not Serializable [1]. This requires a decision on what to do to fix it. This bug is still present in 1.8 RC1 as found by FindBugs [2].
>
> From what I can see the CSVRecord maintains a reference to the CSVParser. This chain of objects maintained in memory is not serializable and leads back to the original input Reader.
>
> I can see from the JApiCmp report that the serial version id was changed for CSVRecord this release so there is still an intention to support serialization. So this should be a blocker.
>
> I could not find a serialisation test in the unit tests for CSVRecord. This quick test added to CSVRecordTest fails:
>
>
> @Test
> public void testSerialization() throws IOException {
> CSVRecord shortRec;
> try (final CSVParser parser = CSVParser.parse("a,b", CSVFormat.newFormat(','))) {
> shortRec = parser.iterator().next();
> }
> final ByteArrayOutputStream out = new ByteArrayOutputStream();
> try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
> oos.writeObject(shortRec);
> }
> }
>
> mvn test -Dtest=CSVRecordTest
>
> [ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
> java.io.NotSerializableException: org.apache.commons.csv.CSVParser
> at org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
>
> If I mark the field csvParser as transient it passes. So this is a problem as raised by FindBugs.
>
>
>
> I also raised [3] the strange implementation of the CSVParser getHeaderNames() which ignores null headers as they cannot be used as a key into the map. However the list of column names could contain the null values. This test currently fails:
>
> @Test
> public void testHeaderNamesWithNull() throws IOException {
> final Reader in = new StringReader("header1,null,header3\n1,2,3\n4,5,6");
> final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
> .withNullString("null")
> .withAllowMissingColumnNames()
> .parse(in).iterator();
> final CSVRecord record = records.next();
> assertEquals(Arrays.asList("header1", null, "header3"), record.getParser().getHeaderNames());
> }
>
> I am not saying it should pass but at least the documentation should state the behaviour in this edge case. That is the list of header names may be shorter than the number of columns when the parser is configured to allow null headers. I’ve not raised a bug ticket for this as it is open to opinion if this is by design or actually a bug. This issue is still present in 1.8 RC1.
>
> Previously I suggested documentation changes for this and another edge case using the header map to be added to the javadoc for getHeaderNames() and getHeaderMap():
>
> - Documentation:
>
> The mapping is only guaranteed to be a one-to-one mapping if the record was created with a format that does not allow duplicate or null header names. Null headers are excluded from the map and duplicates can only map to 1 column.
>
>
> - Bug / Documentation
>
> The CSVParser only stores headers names in a list of header names if they are not null. So the list can be shorter than the number of columns if you use a format that allows empty headers and contains null column names.
>
>
> The ultimate result is that we should document that the purpose of the header names is to provide a list of non-null header names in the order they occur in the header and thus represent keys that can be used in the header map. In certain circumstances there may be more columns in the data than there are header names.
>
>
> Alex
>
>
> [1] https://issues.apache.org/jira/browse/CSV-248 <https://issues.apache.org/jira/browse/CSV-248>
>
> [2] https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html <https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html>
>
> [3] https://markmail.org/message/woti2iymecosihx6 <https://markmail.org/message/woti2iymecosihx6>
>
>
>
> > On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
> >
> > We have fixed quite a few bugs and added some significant enhancements
> > since Apache Commons CSV 1.7 was released, so I would like to release
> > Apache Commons CSV 1.8.
> >
> > Apache Commons CSV 1.8 RC1 is available for review here:
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> > revision 37670)
> >
> > The Git tag commons-csv-1.8-RC1 commit for this RC is
> > c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
> >
> > https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> > You may checkout this tag using:
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> >
> > Maven artifacts are here:
> >
> > https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
> >
> > These are the artifacts and their hashes:
> >
> > #Release SHA-512s
> > #Sat Jan 18 12:01:01 EST 2020
> > commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> > commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> > commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> > commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> > commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> > commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> > commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> > commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
> >
> > I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> > japicmp clean package site deploy' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Additional tests with 'mvn -V clean test' using:
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk1.8.0_241\jre
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-11.0.6
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-12.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\jdk-13.0.2
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-14
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > JaCoCo fails on Java 15-EA because it does not know about class file major
> > version 59:
> >
> > Caused by: java.lang.IllegalArgumentException: Unsupported class file major
> > version 59
> > at
> > org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
> >
> > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > Maven home: C:\Java\apache-maven-3.6.3\bin\..
> > Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> > Files\Java\openjdk\jdk-15
> > Default locale: en_US, platform encoding: Cp1252
> > OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
> >
> > Details of changes since 1.7 are in the release notes:
> >
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
> >
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
> >
> > Site:
> >
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> > (note some *relative* links are broken and the 1.8 directories are not
> > yet created - these will be OK once the site is deployed.)
> >
> > JApiCmp Report (compared to 1.7):
> >
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
> >
> > RAT Report:
> >
> > https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
> >
> > KEYS:
> > https://www.apache.org/dist/commons/KEYS
> >
> > Please review the release candidate and vote.
> > This vote will close no sooner that 72 hours from now.
> >
> > [ ] +1 Release these artifacts
> > [ ] +0 OK, but...
> > [ ] -0 OK, but really should fix...
> > [ ] -1 I oppose this release because...
> >
> > Thank you,
> >
> > Gary Gregory,
> > Release Manager (using key 86fdc7e2a11262cb)
> >
> > For following is intended as a helper and refresher for reviewers.
> >
> > Validating a release candidate
> > ==============================
> >
> > These guidelines are NOT complete.
> >
> > Requirements: Git, Java, Maven.
> >
> > You can validate a release from a release candidate (RC) tag as follows.
> >
> > 1) Clone and checkout the RC tag
> >
> > git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> > commons-csv-1.8-RC1 commons-csv-1.8-RC1
> > cd commons-csv-1.8-RC1
> >
> > 2) Check Apache licenses
> >
> > This step is not required if the site includes a RAT report page which you
> > then must check.
> >
> > mvn apache-rat:check
> >
> > 3) Check binary compatibility
> >
> > Older components still use Apache Clirr:
> >
> > This step is not required if the site includes a Clirr report page which
> > you then must check.
> >
> > mvn clirr:check
> >
> > Newer components use JApiCmp with the japicmp Maven Profile:
> >
> > This step is not required if the site includes a JApiCmp report page which
> > you then must check.
> >
> > mvn install -DskipTests -P japicmp japicmp:cmp
> >
> > 4) Build the package
> >
> > mvn -V clean package
> >
> > You can record the Maven and Java version produced by -V in your VOTE reply.
> > To gather OS information from a command line:
> > Windows: ver
> > Linux: uname -a
> >
> > 5) Build the site for a single module project
> >
> > Note: Some plugins require the components to be installed instead of
> > packaged.
> >
> > mvn site
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > 6) Build the site for a multi-module project
> >
> > mvn site
> > mvn site:stage
> > Check the site reports in:
> > - Windows: target\site\index.html
> > - Linux: target/site/index.html
> >
> > -the end-
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VOTE] Release Apache Commons CSV 1.8 based on RC1
Posted by Alex Herbert <al...@gmail.com>.
Hi Gary,
I raised a few niggles a while back with CSV and the discussion did not receive a response on how to proceed.
There is the major bug CSV-248 where the CSVRecord is not Serializable [1]. This requires a decision on what to do to fix it. This bug is still present in 1.8 RC1 as found by FindBugs [2].
From what I can see the CSVRecord maintains a reference to the CSVParser. This chain of objects maintained in memory is not serializable and leads back to the original input Reader.
I can see from the JApiCmp report that the serial version id was changed for CSVRecord this release so there is still an intention to support serialization. So this should be a blocker.
I could not find a serialisation test in the unit tests for CSVRecord. This quick test added to CSVRecordTest fails:
@Test
public void testSerialization() throws IOException {
CSVRecord shortRec;
try (final CSVParser parser = CSVParser.parse("a,b", CSVFormat.newFormat(','))) {
shortRec = parser.iterator().next();
}
final ByteArrayOutputStream out = new ByteArrayOutputStream();
try (ObjectOutputStream oos = new ObjectOutputStream(out)) {
oos.writeObject(shortRec);
}
}
mvn test -Dtest=CSVRecordTest
[ERROR] testSerialization Time elapsed: 0.032 s <<< ERROR!
java.io.NotSerializableException: org.apache.commons.csv.CSVParser
at org.apache.commons.csv.CSVRecordTest.testSerialization(CSVRecordTest.java:235)
If I mark the field csvParser as transient it passes. So this is a problem as raised by FindBugs.
I also raised [3] the strange implementation of the CSVParser getHeaderNames() which ignores null headers as they cannot be used as a key into the map. However the list of column names could contain the null values. This test currently fails:
@Test
public void testHeaderNamesWithNull() throws IOException {
final Reader in = new StringReader("header1,null,header3\n1,2,3\n4,5,6");
final Iterator<CSVRecord> records = CSVFormat.DEFAULT.withHeader()
.withNullString("null")
.withAllowMissingColumnNames()
.parse(in).iterator();
final CSVRecord record = records.next();
assertEquals(Arrays.asList("header1", null, "header3"), record.getParser().getHeaderNames());
}
I am not saying it should pass but at least the documentation should state the behaviour in this edge case. That is the list of header names may be shorter than the number of columns when the parser is configured to allow null headers. I’ve not raised a bug ticket for this as it is open to opinion if this is by design or actually a bug. This issue is still present in 1.8 RC1.
Previously I suggested documentation changes for this and another edge case using the header map to be added to the javadoc for getHeaderNames() and getHeaderMap():
- Documentation:
The mapping is only guaranteed to be a one-to-one mapping if the record was created with a format that does not allow duplicate or null header names. Null headers are excluded from the map and duplicates can only map to 1 column.
- Bug / Documentation
The CSVParser only stores headers names in a list of header names if they are not null. So the list can be shorter than the number of columns if you use a format that allows empty headers and contains null column names.
The ultimate result is that we should document that the purpose of the header names is to provide a list of non-null header names in the order they occur in the header and thus represent keys that can be used in the header map. In certain circumstances there may be more columns in the data than there are header names.
Alex
[1] https://issues.apache.org/jira/browse/CSV-248 <https://issues.apache.org/jira/browse/CSV-248>
[2] https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html <https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/findbugs.html>
[3] https://markmail.org/message/woti2iymecosihx6 <https://markmail.org/message/woti2iymecosihx6>
> On 18 Jan 2020, at 17:52, Gary Gregory <gg...@apache.org> wrote:
>
> We have fixed quite a few bugs and added some significant enhancements
> since Apache Commons CSV 1.7 was released, so I would like to release
> Apache Commons CSV 1.8.
>
> Apache Commons CSV 1.8 RC1 is available for review here:
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1 (svn
> revision 37670)
>
> The Git tag commons-csv-1.8-RC1 commit for this RC is
> c1c8b32809df295423fc897eae0e8b22bfadfe27 which you can browse here:
>
> https://gitbox.apache.org/repos/asf?p=commons-csv.git;a=commit;h=c1c8b32809df295423fc897eae0e8b22bfadfe27
> You may checkout this tag using:
> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> commons-csv-1.8-RC1 commons-csv-1.8-RC1
>
> Maven artifacts are here:
>
> https://repository.apache.org/content/repositories/orgapachecommons-1489/org/apache/commons/commons-csv/1.8/
>
> These are the artifacts and their hashes:
>
> #Release SHA-512s
> #Sat Jan 18 12:01:01 EST 2020
> commons-csv-1.8-bin.tar.gz=85a876b41aa9ce61f7f533c46df48754e05bddbdef892aed2bac7674b5ea13855de25576364649048dbb55e7fb18a354305b56cb697e85df68a87113954128ed
> commons-csv-1.8-bin.zip=9b86a22367c84a0c96a457e8495f81113b64ae5501eabbe2ea4137654b6baa05bcc24a19626452b80e30ff2dd39214840c6ec534be1db9eec2d12c93eeab2de1
> commons-csv-1.8-javadoc.jar=a481149dfeffe4e915d5d2e846831994223fc7d09ed2b61398c68eed5a672654a141fa6de705aa743d0b5af6fd24a3f4b0d5e7cee238a1f7642673288d4a985d
> commons-csv-1.8-sources.jar=f68e50f8a025a8b2a570b46905b22b5753a83c19bee5c38103d92ec1e47b4e0d27353e7931961e74fe8e67c4909b0ece6ede49a585d2f9180a7a15458b020bc0
> commons-csv-1.8-src.tar.gz=c3268f456978e75c19134e35d05bff77002b2fa7439be2623d58a102cab4f93b0913a1a789f962aafcd677938be1547f47c5dd86e3ea08b7bf8f0420e81beb7a
> commons-csv-1.8-src.zip=ebb32f2406b6afa48472283612c7d0a94f932d7ae7a72ad1d239e2249de12f1e0da7f61d34d95d66b1d1fe95b66b6316af9d1fc93734f610cce4a7163b0900d0
> commons-csv-1.8-test-sources.jar=13508d417e23a5d3f575a39b3aedb20d0d834335d7994f3045fff316e6b12e50cbf9afe908271357b4091d981c178a28dc61bcdb8db60bd0ada07d3de59eacbf
> commons-csv-1.8-tests.jar=901889d4be203c2044df89b7e051d21e7b806e5e56438bf9a7483b334331da94b71de1a129c8bf7967e02479a0922bb834ce37eaabf6662702e147813ecb2b7f
>
> I have tested this with ' mvn -V -Prelease -Ptest-deploy -P jacoco -P
> japicmp clean package site deploy' using:
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\jdk1.8.0_241\jre
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> Additional tests with 'mvn -V clean test' using:
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 1.8.0_241, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\jdk1.8.0_241\jre
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\jdk-11.0.6
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 12.0.2, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\jdk-12.0.2
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 13.0.2, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\jdk-13.0.2
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 14-ea, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\openjdk\jdk-14
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> JaCoCo fails on Java 15-EA because it does not know about class file major
> version 59:
>
> Caused by: java.lang.IllegalArgumentException: Unsupported class file major
> version 59
> at
> org.jacoco.agent.rt.internal_43f5073.asm.ClassReader.<init>(ClassReader.java:195)
>
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: C:\Java\apache-maven-3.6.3\bin\..
> Java version: 15-ea, vendor: Oracle Corporation, runtime: C:\Program
> Files\Java\openjdk\jdk-15
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>
> Details of changes since 1.7 are in the release notes:
>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/RELEASE-NOTES.txt
>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/changes-report.html
>
> Site:
>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/index.html
> (note some *relative* links are broken and the 1.8 directories are not
> yet created - these will be OK once the site is deployed.)
>
> JApiCmp Report (compared to 1.7):
>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/japicmp.html
>
> RAT Report:
>
> https://dist.apache.org/repos/dist/dev/commons/csv/1.8-RC1/site/rat-report.html
>
> KEYS:
> https://www.apache.org/dist/commons/KEYS
>
> Please review the release candidate and vote.
> This vote will close no sooner that 72 hours from now.
>
> [ ] +1 Release these artifacts
> [ ] +0 OK, but...
> [ ] -0 OK, but really should fix...
> [ ] -1 I oppose this release because...
>
> Thank you,
>
> Gary Gregory,
> Release Manager (using key 86fdc7e2a11262cb)
>
> For following is intended as a helper and refresher for reviewers.
>
> Validating a release candidate
> ==============================
>
> These guidelines are NOT complete.
>
> Requirements: Git, Java, Maven.
>
> You can validate a release from a release candidate (RC) tag as follows.
>
> 1) Clone and checkout the RC tag
>
> git clone https://gitbox.apache.org/repos/asf/commons-csv.git --branch
> commons-csv-1.8-RC1 commons-csv-1.8-RC1
> cd commons-csv-1.8-RC1
>
> 2) Check Apache licenses
>
> This step is not required if the site includes a RAT report page which you
> then must check.
>
> mvn apache-rat:check
>
> 3) Check binary compatibility
>
> Older components still use Apache Clirr:
>
> This step is not required if the site includes a Clirr report page which
> you then must check.
>
> mvn clirr:check
>
> Newer components use JApiCmp with the japicmp Maven Profile:
>
> This step is not required if the site includes a JApiCmp report page which
> you then must check.
>
> mvn install -DskipTests -P japicmp japicmp:cmp
>
> 4) Build the package
>
> mvn -V clean package
>
> You can record the Maven and Java version produced by -V in your VOTE reply.
> To gather OS information from a command line:
> Windows: ver
> Linux: uname -a
>
> 5) Build the site for a single module project
>
> Note: Some plugins require the components to be installed instead of
> packaged.
>
> mvn site
> Check the site reports in:
> - Windows: target\site\index.html
> - Linux: target/site/index.html
>
> 6) Build the site for a multi-module project
>
> mvn site
> mvn site:stage
> Check the site reports in:
> - Windows: target\site\index.html
> - Linux: target/site/index.html
>
> -the end-