You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by GitBox <gi...@apache.org> on 2020/06/01 16:29:19 UTC
[GitHub] [commons-io] XenoAmess opened a new pull request #118: refine IOUtils.contentEquals(Reader, Reader)
XenoAmess opened a new pull request #118:
URL: https://github.com/apache/commons-io/pull/118
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/31164655)
Coverage decreased (-0.001%) to 89.718% when pulling **73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 on XenoAmess:refine_contentEquals** into **cd7787277b170d5f439df03850ca0c574566e539 on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-640937650
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/31167650)
Coverage increased (+0.05%) to 89.766% when pulling **a376dd50d6d3d611d51fc764de318587b2434c0b on XenoAmess:refine_contentEquals** into **cd7787277b170d5f439df03850ca0c574566e539 on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465613411
##########
File path: src/test/java/org/apache/commons/io/input/buffer/CircularBufferInputStreamTest.java
##########
@@ -70,6 +70,8 @@ public void testRandomRead() throws Exception {
throw new IllegalStateException("Unexpected random choice value");
}
}
+ bais.close();
+ cbis.close();
}
Review comment:
@sebbASF agreed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/31168325)
Coverage increased (+0.1%) to 89.845% when pulling **fda006373b701538f73f109abaa33ab9e67d81fe on XenoAmess:refine_contentEquals** into **cd7787277b170d5f439df03850ca0c574566e539 on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669180712
> Do you have any proof that using UnsyncBufferedReader etc in contentEquals will result in a slow-down unless further changes are made to contentEquals?
Yes, I ran a rough performance test on a same pc, and the result be time 9e8.(the current version of this pr shows 4.6e8, and master 2.8e9)
> If so, what further changes are needed to contentEquals, and what is the speed improvement?
That trick only be possible when we comparing two(or several) InputReaders.
I shared an index on them, thus the two buffer's index be a same local variable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669137557
Also, the existing prefix we use is Undynchronized, not Unsync.
Gary
On Wed, Aug 5, 2020, 05:48 sebbASF <no...@github.com> wrote:
> AFAICT the new classes UnsyncBufferedReader etc are not actually used
> (except in test code).
>
> I would expect all the contentEquals methods to remain exactly the same
> except for changes to the input sources.
> The logic should remain unchanged (otherwise the new classes are not
> proper replacements).
>
> Likewise I would not expect to see any changes to test cases, only new
> ones for the new classes, and perhaps a few new samples for existing
> methods where the coverage is incomplete.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/commons-io/pull/118#issuecomment-669097163>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJB6N4YDKEPR6TRS52YASDR7ETIJANCNFSM4NP6TZTQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669144943
@sebbASF @garydgregory
> Also, the existing prefix we use is Undynchronized, not Unsync. Gary
> Undynchronized should presumably be Unsynchronized?
>
> However, I think that makes the class names rather long.
> I think Unsync is clear enough as a prefix (and avoids the issue of whether to use -ized or -ised)
I'm neutral on the naming question.
If you get an agreement about it, I'm glad to rename the two classes according to the result.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/36993282)
Coverage increased (+0.1%) to 89.389% when pulling **1f7750a403c6f50e7eae95b1de199067813ab1e7 on xenoamess-fork:refine_contentEquals** into **40b9af504c8c9cb313869786516348775443c086 on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636994263
@garydgregory
> @XenoAmess
>
> * When you create a PR, please do not leave the description empty. It makes it harder to review.
OK
> * If a PR claims a performance improvement, it must back it up; some components do this using JMH microbenchmarks, for example please see Apache Commons BCEL, Crypto, CSV, Lang, Math, RNG.
I have only performance using Jprofiler and I don't know how to embed a test with performance in junit.
I'll learn about JMH microbenchmarks.
thanks.
also, need I create more tests for testing wether this function is correct?
or tests that already exist ,which for the older function, is good enough and I need not create more?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636974841
I also want to change other contentEquals functions in IOUtils (using similar way), but I want to listen to your advices first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] melloware commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
melloware commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462222165
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -790,16 +842,342 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader br1 = toBufferedReader(input1);
- final BufferedReader br2 = toBufferedReader(input2);
- String line1 = br1.readLine();
- String line2 = br2.readLine();
- while (line1 != null && line1.equals(line2)) {
- line1 = br1.readLine();
- line2 = br2.readLine();
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1 = 0;
+ int nowPos2 = 0;
+ int nowRead1;
+ int nowRead2;
+ int nowCheck1 = 0;
+ int nowCheck2 = 0;
+ boolean readEnd1 = false;
+ boolean readEnd2 = false;
+ LastState lastState1 = LastState.newLine;
+ LastState lastState2 = LastState.newLine;
+ while (true) {
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd1) {
+ if (readEnd2) {
+ return true;
+ } else {
+ switch (lastState1) {
+ case r:
+ case newLine:
+ switch (lastState2) {
+ case r:
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState2) {
+ case normal:
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd2) {
+ return true;
+ } else if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ case '\n':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+ } else if (readEnd2) {
+ switch (lastState2) {
+ case r:
+ case newLine:
+ switch (lastState1) {
+ case r:
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState1) {
+ case normal:
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (readEnd1) {
+ return true;
+ } else if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ lastState1 = lastState2 = LastState.r;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck2++;
+ if (readEnd1) {
+ continue;
+ }
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ }
+ continue;
+ default:
+ return false;
+ }
+ case '\n':
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ if (readEnd2) {
+ continue;
+ }
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ }
+ continue;
+ default:
+ if (lastState1 == LastState.r) {
+ lastState1 = LastState.newLine;
+ nowCheck1++;
+ continue;
+ } else {
+ return false;
+ }
+ }
+ default:
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ if (lastState2 == LastState.r) {
+ lastState2 = LastState.newLine;
+ nowCheck2++;
+ continue;
+ } else {
+ return false;
+ }
+ case '\r':
+ return false;
+ default:
+ if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
+ return false;
+ }
+ lastState1 = lastState2 = LastState.normal;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ }
+ }
}
Review comment:
I agree with Gary on this one. In some cases performance gains do not outweigh complexity of the code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-750363553
I will look over all Commons PRs starting tomorrow over the next while I have some time off from work... please be patient.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-639558681
@garydgregory
Hi gary.
please find some time for this pr.
If this pr make sence, then I'll renew other `contentEquals` functions in the same class.
thx.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/36991983)
Coverage increased (+0.3%) to 89.52% when pulling **620bc26385c8a2bed20c70339bbe8dca3faab5ee on xenoamess-fork:refine_contentEquals** into **40b9af504c8c9cb313869786516348775443c086 on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32519545)
Coverage increased (+0.4%) to 90.143% when pulling **93bbde3aea0a724bd9bf32d45d876bdb3fe76779 on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669143244
> AFAICT the new classes UnsyncBufferedReader etc are not actually used (except in test code).
yes, they are not quite used, but used for being parent class of
> I would expect all the contentEquals methods to remain exactly the same except for changes to the input sources.
> The logic should remain unchanged (otherwise the new classes are not proper replacements).
It can, but have huge performance lost.
I also used another trick in those contentEquals functions, which cannot be used if we use that classes.
And if we use the two classes there, will be 1 time slower than the functions we used in this pr (still be about 2 times faster than original)
> Likewise I would not expect to see any changes to test cases, only new ones for the new classes, and perhaps a few new samples for existing methods where the coverage is incomplete.
The changes in test are already moved to another pr, at https://github.com/apache/commons-io/pull/137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465608614
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
Please don't make unrelated changes to test code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/31173743)
Coverage increased (+0.1%) to 89.845% when pulling **af88ba93e4850fa0abcf4fa7d239c8f5d017d1f7 on XenoAmess:refine_contentEquals** into **cd7787277b170d5f439df03850ca0c574566e539 on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32389486)
Coverage decreased (-0.3%) to 89.478% when pulling **66ffa500d25f6781919a68c73f78c720b2969235 on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669143244
@sebbASF
> AFAICT the new classes UnsyncBufferedReader etc are not actually used (except in test code).
yes, they are not quite used, but used for being parent class of
> I would expect all the contentEquals methods to remain exactly the same except for changes to the input sources.
> The logic should remain unchanged (otherwise the new classes are not proper replacements).
It can, but have huge performance lost.
I also used another trick in those contentEquals functions, which cannot be used if we use that classes.
And if we use the two classes there, will be 1 time slower than the functions we used in this pr (still be about 2 times faster than original)
> Likewise I would not expect to see any changes to test cases, only new ones for the new classes, and perhaps a few new samples for existing methods where the coverage is incomplete.
The changes in test are already moved to another pr, at https://github.com/apache/commons-io/pull/137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] eolivelli commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462154657
##########
File path: src/test/java/org/apache/commons/io/performance/IOUtilsContentEqualsPerformanceTest.java
##########
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.performance;
+
+import org.apache.commons.io.IOUtils;
+import org.openjdk.jmh.annotations.*;
+
+import java.io.*;
+import java.util.concurrent.TimeUnit;
+
+import static org.apache.commons.io.IOUtils.EOF;
+import static org.apache.commons.io.IOUtils.toBufferedReader;
+
+/**
+ * Test to show whether using BitSet for removeAll() methods is faster than using HashSet.
+ */
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+@State(Scope.Thread)
+public class IOUtilsContentEqualsPerformanceTest {
+
+ private static final String[] STRINGS = new String[3];
+
+ static {
+ STRINGS[0] = getString0();
+ STRINGS[1] = STRINGS[0] + 'c';
+ STRINGS[2] = STRINGS[0] + 'd';
+ }
+
+ private static final int LOOP = 10;
+
+ @Benchmark
+ public boolean[] testContentEqualsForFileNew() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsNew2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ /**
+ * Old version of IOUtils.contentEquals(Reader, Reader)
+ *
+ * Compares the contents of two Readers to determine if they are equal or
+ * not.
+ * <p>
+ * This method buffers the input internally using
+ * <code>BufferedReader</code> if they are not already buffered.
+ * </p>
+ *
+ * @param input1 the first reader
+ * @param input2 the second reader
+ * @return true if the content of the readers are equal or they both don't
+ * exist, false otherwise
+ * @throws NullPointerException if either input is null
+ * @throws IOException if an I/O error occurs
+ * @since 1.1
+ */
+ @SuppressWarnings("resource")
+ public static boolean contentEqualsOld(final Reader input1, final Reader input2)
+ throws IOException {
+ if (input1 == input2) {
+ return true;
+ }
+ if (input1 == null ^ input2 == null) {
+ return false;
+ }
+ final BufferedReader bufferedInput1 = toBufferedReader(input1);
+ final BufferedReader bufferedInput2 = toBufferedReader(input2);
+
+ int ch = bufferedInput1.read();
+ while (EOF != ch) {
+ final int ch2 = bufferedInput2.read();
+ if (ch != ch2) {
+ return false;
+ }
+ ch = bufferedInput1.read();
+ }
+
+ return bufferedInput2.read() == EOF;
+ }
+
+ public static String getString0() {
+ StringBuilder stringBuilder = new StringBuilder("ab");
+ for (int i = 0; i < 24; i++) {
Review comment:
Got it.
Sorry for the confusion
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465622006
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
@sebbASF OK, I will create a new pr for the changes IOUtilsTestCase.java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-753650101
Hello @XenoAmess
We already have a lot of changes for the next release, so I want to manage expectations such that I would I prefer to get out 3.12 before making even more big changes like these.
But still, let's continue this thread. Starting with the lowest-level bits: we need to justify the addition of the misnamed `Unsync*` classes, the prefix should be `Unsynchronized` like our existing `UnsynchronizedByteArrayInputStream`, which I've already mentioned.
We need performances test that show the differences, if any, between the JRE's classes and our proposed `Unsynchronized` versions. Since you propose two such classes `UnsyncBufferedInputStream` and `UnsyncBufferedReader`, that's two new tests. Or did I miss these here?
I think you should create a new PR for just these two new classes and their tests. This will make the work simpler for everyone when reviewing and testing.
TY.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32473623)
Coverage decreased (-0.5%) to 89.271% when pulling **a8da767a03973a5a2d126b9ff6da9f3063eeba2b on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-665831472
Hi.
I just found a bug in contentEqualsIgnoreEOLNew2.
Will redo it tomorrow.
-----------
Actually will redo it anyway, as I thought it good to split it out to some class structure, rather than repeat two times in the function.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/37260076)
Coverage increased (+0.005%) to 89.4% when pulling **eb25f9a554b8302b5bc870c6b0c960bd74a2f45c on xenoamess-fork:refine_contentEquals** into **c54bf688e94b550c3ccd4c0789e0b3e00cf1d0ea on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465622006
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
@sebbASF OK, I will create a new pr for the changes in IOUtilsTestCase.java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-665587079
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465623190
##########
File path: src/main/java/org/apache/commons/io/input/buffer/UnsyncBufferedInputStream.java
##########
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.input.buffer;
+
+import java.io.IOException;
+import java.io.InputStream;
+import org.apache.commons.io.IOUtils;
+import static org.apache.commons.io.IOUtils.EOF;
+
+/**
+ * A BufferedReader class who does not care about thread safety, but very much faster.
+ *
+ * Should be able to replace java.io.BufferedReader in nearly every use-cases when you
+ * need the Reader be buffered, but do not need it have thread safety.
+ */
+public class UnsyncBufferedInputStream extends InputStream {
+ protected final InputStream inputStream;
+ protected final byte[] byteBuffer;
+
+ protected int nowIndex = 0;
+ protected int nowLimit = 0;
+
Review comment:
@sebbASF
Hi.
for the protected fields:
OK, I can change them to private.
for the public functions:
they are already as less as can.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465659496
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
@sebbASF
pr created at
https://github.com/apache/commons-io/pull/137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669180712
@sebbASF
> Do you have any proof that using UnsyncBufferedReader etc in contentEquals will result in a slow-down unless further changes are made to contentEquals?
Yes, I ran a rough performance test on a same pc, and the result be time 9e8.(the current version of this pr shows 4.6e8, and master 2.8e9)
If you need a full detailed performance test I can find some time to run one.
> If so, what further changes are needed to contentEquals, and what is the speed improvement?
That trick only be possible when we comparing two(or several) InputReaders.
I shared an index on them, thus the two buffer's index be a same local variable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465612559
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
> Please don't make unrelated changes to test code.
@sebbASF but the original tests are too loose, it did not cover some situations.
in other words, codes which can pass the original tests might be wrong.
So I added some test codes.
If you want me to move them to a seperate new file, I'll do it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-745358426
> I think would like to/will soon-ish bring in the underlying `Unsynchronized*` classes first since they seem like useful general building blocks. Then this PR can be rebased. I would like to do this work at some point this week or at the weekend. Stay tuned.
@garydgregory got it. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669170277
> Undynchronized should presumably be Unsynchronized?
Phone keyboard! Right, today we have:
- org.apache.commons.io.input.UnsynchronizedByteArrayInputStream
- org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream
>
> However, I think that makes the class names rather long.
> I think Unsync is clear enough as a prefix (and avoids the issue of whether to use -ized or -ised)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-665831472
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636971532
need I make a performance test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636994263
> @XenoAmess
>
> * When you create a PR, please do not leave the description empty. It makes it harder to review.
OK
> * If a PR claims a performance improvement, it must back it up; some components do this using JMH microbenchmarks, for example please see Apache Commons BCEL, Crypto, CSV, Lang, Math, RNG.
I have only performance using Jprofiler and I don't know how to embed a test with performance in junit.
I'll learn about JMH microbenchmarks.
thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-665748884
It occurs to me that the code is effectively buffering the output from a BufferedReader (or BufferedInputStream).
One would expect these classes to be reasonably fast, however the JVM has to do locking and other checks in order to support multi-threading. It has to do this for each read() call.
Rather than implement the buffering directly in the compare methods, it might be better to implement a generic, non-threadsafe buffered reader/stream that can be used in situations such as these. [No need to wrap the original input in a buffered version first]. It would be a non-threadsafe version of BufferedReader/InputStream.
The original code should then work without any change other than to add the filter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669143244
@sebbASF
> AFAICT the new classes UnsyncBufferedReader etc are not actually used (except in test code).
yes, they are not quite used, but used for being parent class of LineEndUnifiedBufferedReader
I also have a plan to use them insteadof BufferedReader in some other places in this repo.
But this pr is already large, and I don't think it good to change too much more things.
> I would expect all the contentEquals methods to remain exactly the same except for changes to the input sources.
> The logic should remain unchanged (otherwise the new classes are not proper replacements).
It can, but have huge performance lost.
I also used another trick in those contentEquals functions, which cannot be used if we use that classes.
And if we use the two classes there, will be 1 time slower than the functions we used in this pr (still be about 2 times faster than original)
> Likewise I would not expect to see any changes to test cases, only new ones for the new classes, and perhaps a few new samples for existing methods where the coverage is incomplete.
The changes in test are already moved to another pr, at https://github.com/apache/commons-io/pull/137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-665609575
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465612559
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
> Please don't make unrelated changes to test code.
@sebbASF but the original tests are too loose, it did not cover some situations.
in other words, codes which can pass the original tests might be wrong.
So I added some test codes.
If you want me to move them to another file, I'll do it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-665609575
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465638699
##########
File path: src/main/java/org/apache/commons/io/input/buffer/UnsyncBufferedInputStream.java
##########
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.input.buffer;
+
+import java.io.IOException;
+import java.io.InputStream;
+import org.apache.commons.io.IOUtils;
+import static org.apache.commons.io.IOUtils.EOF;
+
+/**
+ * A BufferedReader class who does not care about thread safety, but very much faster.
+ *
+ * Should be able to replace java.io.BufferedReader in nearly every use-cases when you
+ * need the Reader be buffered, but do not need it have thread safety.
+ */
+public class UnsyncBufferedInputStream extends InputStream {
+ protected final InputStream inputStream;
+ protected final byte[] byteBuffer;
+
+ protected int nowIndex = 0;
+ protected int nowLimit = 0;
+
Review comment:
AFAICT UnsyncBufferedInputStream does not need to expose any getter/setter methods, not does it need to expose eat().
As to UnsyncBufferedReader, AFAICT LineEndUnifiedBufferedReader could be rewritten to act as a Filter which would then make it more versatile.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669154882
Do you have any proof that using UnsyncBufferedReader etc in contentEquals will result in a slow-down unless further changes are made to contentEquals?
If so, what further changes are needed to contentEquals, and what is the speed improvement?
As to any renaming, that can be done later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462250040
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -882,15 +894,26 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
nowPos2 += nowRead2;
}
}
+
if (readEnd1) {
+ // if input1 ended.
Review comment:
This comment is not necessary comment if readEnd1 is properly commented
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -814,9 +814,21 @@ public static boolean contentEquals(final Reader input1, final Reader input2)
}
private enum LastState {
- r,
- normal,
- newLine;
+ /**
+ * If last char is '\r'.
+ */
+ R,
+
+ /**
+ * If last char is not '\r' nor '\n'.
+ */
+ NORMAL,
+
+ /**
+ * If we just moved to a new line.
+ * It might sounds weird but after you see the codes you can know it.
+ */
+ NEW_LINE;
Review comment:
This could be NL to agree with CR above
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -814,9 +814,21 @@ public static boolean contentEquals(final Reader input1, final Reader input2)
}
private enum LastState {
- r,
- normal,
- newLine;
+ /**
+ * If last char is '\r'.
+ */
+ R,
+
+ /**
+ * If last char is not '\r' nor '\n'.
+ */
+ NORMAL,
+
+ /**
+ * If we just moved to a new line.
+ * It might sounds weird but after you see the codes you can know it.
Review comment:
This comment is not useful
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -882,15 +894,26 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
nowPos2 += nowRead2;
}
}
+
if (readEnd1) {
+ // if input1 ended.
if (readEnd2) {
+ // if both input1 and input2 ended here, we just return true.
return true;
} else {
+ // if input2 not ended.
switch (lastState1) {
- case r:
- case newLine:
+ case R:
+ // if last state of input1 is "\r"
Review comment:
Unnecessary if the enum value has a sensible name and comment
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -1147,31 +1201,43 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
}
continue;
default:
- if (lastState1 == LastState.r) {
- lastState1 = LastState.newLine;
+ // if input2's next is normal.
+ // if input1's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState1 == LastState.R) {
+ lastState1 = LastState.NEW_LINE;
nowCheck1++;
continue;
} else {
return false;
}
}
default:
+ // if input1's next is normal.
switch (charArray2[nowCheck2]) {
case '\n':
- if (lastState2 == LastState.r) {
- lastState2 = LastState.newLine;
+ // if input2's next is '\n'.
+ // if input2's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState2 == LastState.R) {
+ lastState2 = LastState.NEW_LINE;
nowCheck2++;
continue;
} else {
return false;
}
case '\r':
+ // if input2's next is '\r'.
+ // illegal.
return false;
default:
+ // if input2's next is normal.
+ // if equal then legal.
+ // otherwise illegal.
if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
return false;
}
- lastState1 = lastState2 = LastState.normal;
+ lastState1 = lastState2 = LastState.NORMAL;
nowCheck1++;
nowCheck2++;
continue;
Review comment:
Most of the comments above are unnecessary if the variables are well named and commented.
What is missing is how the algorithm works, and what the various combinations of state actually mean in terms of the algorithm.
I'm not clear why the code sometimes checks for '\r' and sometimes uses the enum.
Why not use a variable containing the last character?
==
I suspect the code could be much simplified by using a suitable filter on the inputs.
There are several examples in IO, and NET has FromNetASCIIInputStream which deals with CRLF conversions
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -882,15 +894,26 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
nowPos2 += nowRead2;
}
}
+
Review comment:
To prevent accidental run-away, it would be safer to use >= to compare incremented indexes
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -814,9 +814,21 @@ public static boolean contentEquals(final Reader input1, final Reader input2)
}
private enum LastState {
- r,
- normal,
- newLine;
+ /**
+ * If last char is '\r'.
+ */
+ R,
Review comment:
Why is this not called CR or CARRIAGE_RETURN?
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -882,15 +894,26 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
nowPos2 += nowRead2;
}
}
+
Review comment:
If the index starts at a value below the limit, and you increment it by 1 between each check, then I agree that the values must be equal at some point, assuming single-threaded code.
However if there is a bug in the code that can cause the index to be incremented twice, it may never match.
Comparison using >= avoids this potential error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-678616521
Hi.
Is there anything that I should do/fix for this pr?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636971532
need I make a performance test case to show the performance?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465608459
##########
File path: src/test/java/org/apache/commons/io/input/buffer/CircularBufferInputStreamTest.java
##########
@@ -70,6 +70,8 @@ public void testRandomRead() throws Exception {
throw new IllegalStateException("Unexpected random choice value");
}
}
+ bais.close();
+ cbis.close();
}
Review comment:
This should be the subject of a separate PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-668293267
The original comparison methods are easy to follow.
It would be good to keep the same clarity by defining new versions of BufferedReader that have good single-threaded performance. Such classes would also be useful elsewhere.
So for example
BufferedReader bufferedInput1 = toBufferedReader(input1);
would become something like:
FastBufferedReader bufferedInput1 = toFastBuffer(input1) (*)
where toFastBuffer() works like buffer() but returns an instance of a non-threadsafe FastBufferedReader.
The FastBufferedxxx classes would also be useful in their own right elsewhere in IO.
Obviously the documentation would need to make it clear that they are not thread-safe.
I suspect they don't need to implement mark (which would make them simpler).
(*) FastBuffer is just a suggestion for the name. Maybe QandDBuffer would be better (Quick and Dirty)!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-744519182
I think would like to/will soon-ish bring in the underlying `Unsynchronized*` classes first since they seem like useful general building blocks. Then this PR can be rebased. I would like to do this work at some point this week or at the weekend. Stay tuned.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32474013)
Coverage increased (+0.3%) to 90.008% when pulling **67e0d3d880a66acce4d054e09fe4c13607ff94ab on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-758804757
@garydgregory unsynced buffered classes deleted.
re-run performance tests, results at https://pastebin.ubuntu.com/p/ZpZFHRMQkh/
please do a re-review.
Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636996577
@garydgregory
I will use this as example and change its content and add the test into this repo if you don't mind.
https://github.com/apache/commons-lang/blob/5cdac9cfd5a74b0a52ebde32798b973c6edbaa79/src/test/java/org/apache/commons/lang3/HashSetvBitSetTest.java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-749616807
> @garydgregory Weekend now. Any news？:)
ping? :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-755419401
> Hello @XenoAmess
> We already have a lot of changes for the next release, so I want to manage expectations such that I would I prefer to get out 3.12 before making even more big changes like these.
>
> But still, let's continue this thread. Starting with the lowest-level bits: we need to justify the addition of the misnamed `Unsync*` classes, the prefix should be `Unsynchronized` like our existing `UnsynchronizedByteArrayInputStream`, which I've already mentioned. Perhaps our addition of UnsynchronizedByteArrayInputStream was a mistake since the Java folks make it sounds like these are superfluous in https://bugs.openjdk.java.net/browse/JDK-4097272, so I think we need to see a performance test that shows there is a clear performance benefit to adding those as valuable on their own.
>
> We need performances test that show the differences, if any, between the JRE's classes and our proposed `Unsynchronized` versions. Since you propose two such classes `UnsyncBufferedInputStream` and `UnsyncBufferedReader`, that's two new tests. Or did I miss these here?
>
> I think you should create a new PR for just these two new classes and their tests. This will make the work simpler for everyone when reviewing and testing.
>
> TY.
@garydgregory Hi. I done the performance test at https://github.com/apache/commons-io/pull/184.
The two classes is faster, but not as that faster as we thought.
Detailed performance tests results: https://pastebin.ubuntu.com/p/TtDxYw4WVG/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-637035997
@garydgregory
Performance test done.
**Conclusion:**
In cases for small content readers, the new function is slightly faster.
but in cases for large content, the new function can be 6.5 times faster.
```
D:\workspace\commons-io>mvn -Pbenchmark
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< commons-io:commons-io >------------------------
[INFO] Building Apache Commons IO 2.7.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ commons-io ---
[INFO] Deleting D:\workspace\commons-io\target
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[copy] Copying 2 files to D:\workspace\commons-io\target\apidocs\META-INF
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ commons-io ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ commons-io ---
[INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] Working directory: D:\workspace\commons-io
[INFO] Storing buildNumber: 73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 at timestamp: 1591034434847
[INFO] Storing buildScmBranch: refine_contentEquals
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\workspace\commons-io\src\main\resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\classes
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Recompile with -Xlint:deprecation for details.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOException
List.java uses unchecked or unsafe operations.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-bundle-plugin:4.2.1:manifest (bundle-manifest) @ commons-io ---
[INFO]
[INFO] --- animal-sniffer-maven-plugin:1.18:check (checkAPIcompatibility) @ commons-io ---
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java18:1.0
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:158: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:158: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:125: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.flip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:127: Covariant return type change detected: java.nio.Buffer java.
nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:207: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:209: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.flip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:213: Covariant return type change detected: java.nio.Buffer java.
nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:74: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:128: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:222: Covariant return type change detected: java.nio.Buffer
java.nio.CharBuffer.mark() has been changed to java.nio.CharBuffer java.nio.CharBuffer.mark()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:223: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.mark() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.mark()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:245: Covariant return type change detected: java.nio.Buffer
java.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:246: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.rewind() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:247: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:249: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.rewind() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:250: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:258: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.position(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:283: Covariant return type change detected: java.nio.Buffer jav
a.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:309: Covariant return type change detected: java.nio.Buffer jav
a.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:331: Covariant return type change detected: java.nio.Buffer jav
a.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:340: Covariant return type change detected: java.nio.Buffer jav
a.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:987: Covariant return type change detected: java.nio.Buffer java.nio.CharBuffer.f
lip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:2008: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffer.
position(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:2009: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffer.
limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] Copying 45 resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\test-classes
[INFO] /D:/workspace/commons-io/src/test/java/org/apache/commons/io/output/FileWriterWithEncodingTest.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/test/java/org/apache/commons/io/output/FileWriterWithEncodingTest.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (prepare-agent) @ commons-io ---
[INFO] argLine set to -javaagent:C:\\Users\\xenoa\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.5\\org.jacoco.agent-0.8.5-runtime.jar=destfile=D:\\workspa
ce\\commons-io\\target\\jacoco.exec
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ commons-io ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:exec (benchmark) @ commons-io ---
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/C:/Users/xenoa/.m2/repository/org/openjdk/jmh/jmh-core/1.21/jmh-core-1.21.jar) to field j
ava.io.PrintStream.charOut
WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew
# Run progress: 0.00% complete, ETA 00:33:20
# Fork: 1 of 5
# Warmup Iteration 1: 8403699.664 ns/op
# Warmup Iteration 2: 8342097.081 ns/op
# Warmup Iteration 3: 7431033.061 ns/op
# Warmup Iteration 4: 7714089.746 ns/op
# Warmup Iteration 5: 7338344.241 ns/op
Iteration 1: 7459707.308 ns/op
Iteration 2: 8792593.849 ns/op
Iteration 3: 10726891.104 ns/op
Iteration 4: 8550254.786 ns/op
Iteration 5: 8001410.312 ns/op
# Run progress: 5.00% complete, ETA 00:31:55
# Fork: 2 of 5
# Warmup Iteration 1: 7848885.804 ns/op
# Warmup Iteration 2: 7489775.075 ns/op
# Warmup Iteration 3: 7328874.652 ns/op
# Warmup Iteration 4: 7308097.663 ns/op
# Warmup Iteration 5: 7298969.001 ns/op
Iteration 1: 7347818.649 ns/op
Iteration 2: 7251497.464 ns/op
Iteration 3: 7366967.231 ns/op
Iteration 4: 7420473.516 ns/op
Iteration 5: 7517702.630 ns/op
# Run progress: 10.00% complete, ETA 00:30:14
# Fork: 3 of 5
# Warmup Iteration 1: 7403000.592 ns/op
# Warmup Iteration 2: 7237561.577 ns/op
# Warmup Iteration 3: 7328035.212 ns/op
# Warmup Iteration 4: 7310369.686 ns/op
# Warmup Iteration 5: 7258066.255 ns/op
Iteration 1: 7387172.526 ns/op
Iteration 2: 7278674.400 ns/op
Iteration 3: 7307894.887 ns/op
Iteration 4: 7363340.103 ns/op
Iteration 5: 7241780.811 ns/op
# Run progress: 15.00% complete, ETA 00:28:33
# Fork: 4 of 5
# Warmup Iteration 1: 7549794.264 ns/op
# Warmup Iteration 2: 7209186.671 ns/op
# Warmup Iteration 3: 7680527.859 ns/op
# Warmup Iteration 4: 7870276.336 ns/op
# Warmup Iteration 5: 8008019.376 ns/op
Iteration 1: 7868122.956 ns/op
Iteration 2: 8014526.603 ns/op
Iteration 3: 7610742.998 ns/op
Iteration 4: 7339908.804 ns/op
Iteration 5: 7466605.970 ns/op
# Run progress: 20.00% complete, ETA 00:26:52
# Fork: 5 of 5
# Warmup Iteration 1: 8062262.933 ns/op
# Warmup Iteration 2: 7888885.647 ns/op
# Warmup Iteration 3: 8092374.353 ns/op
# Warmup Iteration 4: 7535815.813 ns/op
# Warmup Iteration 5: 8376873.032 ns/op
Iteration 1: 7641931.603 ns/op
Iteration 2: 8134969.268 ns/op
Iteration 3: 8823474.603 ns/op
Iteration 4: 9548556.489 ns/op
Iteration 5: 8888150.622 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew":
7934046.780 ?99.9%) 644395.868 ns/op [Average]
(min, avg, max) = (7241780.811, 7934046.780, 10726891.104), stdev = 860250.047
CI (99.9%): [7289650.912, 8578442.648] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2
# Run progress: 25.00% complete, ETA 00:25:12
# Fork: 1 of 5
# Warmup Iteration 1: 427439412.500 ns/op
# Warmup Iteration 2: 411941108.000 ns/op
# Warmup Iteration 3: 415405856.000 ns/op
# Warmup Iteration 4: 408246064.000 ns/op
# Warmup Iteration 5: 381986007.407 ns/op
Iteration 1: 372559803.704 ns/op
Iteration 2: 372221244.444 ns/op
Iteration 3: 394561276.923 ns/op
Iteration 4: 410318660.000 ns/op
Iteration 5: 417270562.500 ns/op
# Run progress: 30.00% complete, ETA 00:23:36
# Fork: 2 of 5
# Warmup Iteration 1: 385519323.077 ns/op
# Warmup Iteration 2: 341417310.000 ns/op
# Warmup Iteration 3: 337620636.667 ns/op
# Warmup Iteration 4: 337741280.000 ns/op
# Warmup Iteration 5: 322947274.194 ns/op
Iteration 1: 334579670.000 ns/op
Iteration 2: 324313664.516 ns/op
Iteration 3: 349864203.448 ns/op
Iteration 4: 345000172.414 ns/op
Iteration 5: 338797430.000 ns/op
# Run progress: 35.00% complete, ETA 00:21:56
# Fork: 3 of 5
# Warmup Iteration 1: 424080325.000 ns/op
# Warmup Iteration 2: 409074288.000 ns/op
# Warmup Iteration 3: 397216476.923 ns/op
# Warmup Iteration 4: 379384881.481 ns/op
# Warmup Iteration 5: 406599224.000 ns/op
Iteration 1: 344006786.667 ns/op
Iteration 2: 318393962.500 ns/op
Iteration 3: 326383164.516 ns/op
Iteration 4: 326728512.903 ns/op
Iteration 5: 339791536.667 ns/op
# Run progress: 40.00% complete, ETA 00:20:17
# Fork: 4 of 5
# Warmup Iteration 1: 359791750.000 ns/op
# Warmup Iteration 2: 344011470.000 ns/op
# Warmup Iteration 3: 317443868.750 ns/op
# Warmup Iteration 4: 366668921.429 ns/op
# Warmup Iteration 5: 340980226.667 ns/op
Iteration 1: 362224464.286 ns/op
Iteration 2: 329631980.645 ns/op
Iteration 3: 328959990.323 ns/op
Iteration 4: 331093680.645 ns/op
Iteration 5: 328042438.710 ns/op
# Run progress: 45.00% complete, ETA 00:18:37
# Fork: 5 of 5
# Warmup Iteration 1: 328533070.968 ns/op
# Warmup Iteration 2: 321877478.125 ns/op
# Warmup Iteration 3: 315723571.875 ns/op
# Warmup Iteration 4: 323838003.226 ns/op
# Warmup Iteration 5: 325582848.387 ns/op
Iteration 1: 322872964.516 ns/op
Iteration 2: 320972165.625 ns/op
Iteration 3: 322146753.125 ns/op
Iteration 4: 320983556.250 ns/op
Iteration 5: 332471216.129 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2":
344567594.458 ?99.9%) 21104733.591 ns/op [Average]
(min, avg, max) = (318393962.500, 344567594.458, 417270562.500), stdev = 28174215.532
CI (99.9%): [323462860.868, 365672328.049] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld
# Run progress: 50.00% complete, ETA 00:16:57
# Fork: 1 of 5
# Warmup Iteration 1: 8040772.910 ns/op
# Warmup Iteration 2: 7850546.468 ns/op
# Warmup Iteration 3: 8366507.191 ns/op
# Warmup Iteration 4: 8919272.193 ns/op
# Warmup Iteration 5: 8934492.411 ns/op
Iteration 1: 9849140.059 ns/op
Iteration 2: 10280016.496 ns/op
Iteration 3: 9377572.540 ns/op
Iteration 4: 8634857.550 ns/op
Iteration 5: 8796502.111 ns/op
# Run progress: 55.00% complete, ETA 00:15:14
# Fork: 2 of 5
# Warmup Iteration 1: 9196890.441 ns/op
# Warmup Iteration 2: 8651530.769 ns/op
# Warmup Iteration 3: 9130613.504 ns/op
# Warmup Iteration 4: 9528571.075 ns/op
# Warmup Iteration 5: 8603469.046 ns/op
Iteration 1: 8649056.612 ns/op
Iteration 2: 8740562.707 ns/op
Iteration 3: 8636735.807 ns/op
Iteration 4: 8658414.545 ns/op
Iteration 5: 9077745.554 ns/op
# Run progress: 60.00% complete, ETA 00:13:32
# Fork: 3 of 5
# Warmup Iteration 1: 9210640.074 ns/op
# Warmup Iteration 2: 7950668.124 ns/op
# Warmup Iteration 3: 8049352.132 ns/op
# Warmup Iteration 4: 7906520.949 ns/op
# Warmup Iteration 5: 8193835.545 ns/op
Iteration 1: 7833574.706 ns/op
Iteration 2: 8025837.640 ns/op
Iteration 3: 8621776.034 ns/op
Iteration 4: 8516147.915 ns/op
Iteration 5: 8379908.710 ns/op
# Run progress: 65.00% complete, ETA 00:11:50
# Fork: 4 of 5
# Warmup Iteration 1: 8810415.141 ns/op
# Warmup Iteration 2: 8794569.859 ns/op
# Warmup Iteration 3: 8730751.745 ns/op
# Warmup Iteration 4: 8563348.202 ns/op
# Warmup Iteration 5: 8946487.757 ns/op
Iteration 1: 8798953.725 ns/op
Iteration 2: 9077297.461 ns/op
Iteration 3: 9203544.250 ns/op
Iteration 4: 10801144.492 ns/op
Iteration 5: 10137045.694 ns/op
# Run progress: 70.00% complete, ETA 00:10:09
# Fork: 5 of 5
# Warmup Iteration 1: 10034710.933 ns/op
# Warmup Iteration 2: 7852066.954 ns/op
# Warmup Iteration 3: 8036246.265 ns/op
# Warmup Iteration 4: 8208804.184 ns/op
# Warmup Iteration 5: 8919215.241 ns/op
Iteration 1: 7703625.019 ns/op
Iteration 2: 8173784.069 ns/op
Iteration 3: 8476930.000 ns/op
Iteration 4: 9254641.073 ns/op
Iteration 5: 8732920.419 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld":
8897509.408 ?99.9%) 558227.023 ns/op [Average]
(min, avg, max) = (7703625.019, 8897509.408, 10801144.492), stdev = 745217.105
CI (99.9%): [8339282.385, 9455736.431] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2
# Run progress: 75.00% complete, ETA 00:08:27
# Fork: 1 of 5
# Warmup Iteration 1: 2274597420.000 ns/op
# Warmup Iteration 2: 2161414140.000 ns/op
# Warmup Iteration 3: 2060648900.000 ns/op
# Warmup Iteration 4: 2034553880.000 ns/op
# Warmup Iteration 5: 2036716060.000 ns/op
Iteration 1: 2039811500.000 ns/op
Iteration 2: 2029332780.000 ns/op
Iteration 3: 2048828760.000 ns/op
Iteration 4: 2167898680.000 ns/op
Iteration 5: 2375598240.000 ns/op
# Run progress: 80.00% complete, ETA 00:06:47
# Fork: 2 of 5
# Warmup Iteration 1: 2302730460.000 ns/op
# Warmup Iteration 2: 2169550200.000 ns/op
# Warmup Iteration 3: 2183069220.000 ns/op
# Warmup Iteration 4: 2219912760.000 ns/op
# Warmup Iteration 5: 2276130160.000 ns/op
Iteration 1: 2256782140.000 ns/op
Iteration 2: 2212519940.000 ns/op
Iteration 3: 2194957540.000 ns/op
Iteration 4: 2139474860.000 ns/op
Iteration 5: 2239168680.000 ns/op
# Run progress: 85.00% complete, ETA 00:05:07
# Fork: 3 of 5
# Warmup Iteration 1: 2871005625.000 ns/op
# Warmup Iteration 2: 2234621780.000 ns/op
# Warmup Iteration 3: 2057113240.000 ns/op
# Warmup Iteration 4: 2106691240.000 ns/op
# Warmup Iteration 5: 2239427440.000 ns/op
Iteration 1: 2221194720.000 ns/op
Iteration 2: 2252754040.000 ns/op
Iteration 3: 2362948520.000 ns/op
Iteration 4: 2233297260.000 ns/op
Iteration 5: 2289872180.000 ns/op
# Run progress: 90.00% complete, ETA 00:03:25
# Fork: 4 of 5
# Warmup Iteration 1: 2316297000.000 ns/op
# Warmup Iteration 2: 2186447840.000 ns/op
# Warmup Iteration 3: 2186496100.000 ns/op
# Warmup Iteration 4: 2210486500.000 ns/op
# Warmup Iteration 5: 2248586640.000 ns/op
Iteration 1: 2228412480.000 ns/op
Iteration 2: 2293236620.000 ns/op
Iteration 3: 2300933160.000 ns/op
Iteration 4: 2050526440.000 ns/op
Iteration 5: 2103876620.000 ns/op
# Run progress: 95.00% complete, ETA 00:01:43
# Fork: 5 of 5
# Warmup Iteration 1: 2231988940.000 ns/op
# Warmup Iteration 2: 2132075180.000 ns/op
# Warmup Iteration 3: 2107424340.000 ns/op
# Warmup Iteration 4: 2244625860.000 ns/op
# Warmup Iteration 5: 2212587240.000 ns/op
Iteration 1: 2358689240.000 ns/op
Iteration 2: 2360920240.000 ns/op
Iteration 3: 2304193360.000 ns/op
Iteration 4: 2120541520.000 ns/op
Iteration 5: 2225049120.000 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2":
2216432745.600 ?99.9%) 79521556.878 ns/op [Average]
(min, avg, max) = (2029332780.000, 2216432745.600, 2375598240.000), stdev = 106159003.301
CI (99.9%): [2136911188.722, 2295954302.478] (assumes normal distribution)
# Run complete. Total time: 00:34:37
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew avgt 25 7934046.780 ? 644395.868 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2 avgt 25 344567594.458 ?21104733.591 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsOld avgt 25 8897509.408 ? 558227.023 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2 avgt 25 2216432745.600 ?79521556.878 ns/op
Benchmark result is saved to target/jmh-result.org.apache.json
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.8.2:attach-descriptor (attach-descriptor) @ commons-io ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:test-jar (default) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.0:jar-no-fork (create-source-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-sources.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.0:test-jar-no-fork (create-source-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-test-sources.jar
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:check (check) @ commons-io ---
[INFO] Skipping JaCoCo execution due to missing execution data file:D:\workspace\commons-io\target\jacoco.exec
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (default-cli) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] >>> clirr-maven-plugin:2.8:check (default-cli) > compile @ commons-io >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ commons-io ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ commons-io ---
[INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] Working directory: D:\workspace\commons-io
[INFO] Storing buildNumber: 73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 at timestamp: 1591036535724
[INFO] Storing buildScmBranch: refine_contentEquals
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\workspace\commons-io\src\main\resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\classes
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Recompile with -Xlint:deprecation for details.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOException
List.java uses unchecked or unsafe operations.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] <<< clirr-maven-plugin:2.8:check (default-cli) < compile @ commons-io <<<
[INFO]
[INFO]
[INFO] --- clirr-maven-plugin:2.8:check (default-cli) @ commons-io ---
[INFO] Comparing to version: 2.7
[INFO] Succeeded with 0 errors; 0 warnings; and 0 other changes.
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.0:check (default-cli) @ commons-io ---
[INFO]
[INFO] >>> maven-javadoc-plugin:3.1.1:javadoc (default-cli) > generate-sources @ commons-io >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] <<< maven-javadoc-plugin:3.1.1:javadoc (default-cli) < generate-sources @ commons-io <<<
[INFO]
[INFO]
[INFO] --- maven-javadoc-plugin:3.1.1:javadoc (default-cli) @ commons-io ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 35:31 min
[INFO] Finished at: 2020-06-02T02:35:59+08:00
[INFO] ------------------------------------------------------------------------
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669180712
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] eolivelli commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462045891
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -752,19 +777,46 @@ public static boolean contentEquals(final Reader input1, final Reader input2)
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader bufferedInput1 = toBufferedReader(input1);
- final BufferedReader bufferedInput2 = toBufferedReader(input2);
- int ch = bufferedInput1.read();
- while (EOF != ch) {
- final int ch2 = bufferedInput2.read();
- if (ch != ch2) {
- return false;
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1;
+ int nowPos2;
+ int nowRead1;
+ int nowRead2;
+ while (true) {
+ nowPos1 = 0;
+ nowPos2 = 0;
+ for (int nowCheck = 0; nowCheck < CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE; nowCheck++) {
+ if (nowPos1 == nowCheck) {
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ return nowPos2 == nowCheck && input2.read() == -1;
+ }
+ nowPos1 += nowRead1;
+ }
+ if (nowPos2 == nowCheck) {
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ return nowPos1 == nowCheck && input1.read() == -1;
+ }
+ nowPos2 += nowRead2;
+ }
+ if (charArray1[nowCheck] != charArray2[nowCheck]) {
+ return false;
+ }
}
- ch = bufferedInput1.read();
}
+ }
- return bufferedInput2.read() == EOF;
+ private enum LastState {
+ r,
Review comment:
Can you please use UPPERCASE identifiers?
Also, IMHO it is better to give meaningful names or add a minimal explanation
##########
File path: src/test/java/org/apache/commons/io/performance/IOUtilsContentEqualsPerformanceTest.java
##########
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.performance;
+
+import org.apache.commons.io.IOUtils;
+import org.openjdk.jmh.annotations.*;
+
+import java.io.*;
+import java.util.concurrent.TimeUnit;
+
+import static org.apache.commons.io.IOUtils.EOF;
+import static org.apache.commons.io.IOUtils.toBufferedReader;
+
+/**
+ * Test to show whether using BitSet for removeAll() methods is faster than using HashSet.
+ */
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+@State(Scope.Thread)
+public class IOUtilsContentEqualsPerformanceTest {
+
+ private static final String[] STRINGS = new String[3];
+
+ static {
+ STRINGS[0] = getString0();
+ STRINGS[1] = STRINGS[0] + 'c';
+ STRINGS[2] = STRINGS[0] + 'd';
+ }
+
+ private static final int LOOP = 10;
+
+ @Benchmark
+ public boolean[] testContentEqualsForFileNew() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsNew2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ /**
+ * Old version of IOUtils.contentEquals(Reader, Reader)
+ *
+ * Compares the contents of two Readers to determine if they are equal or
+ * not.
+ * <p>
+ * This method buffers the input internally using
+ * <code>BufferedReader</code> if they are not already buffered.
+ * </p>
+ *
+ * @param input1 the first reader
+ * @param input2 the second reader
+ * @return true if the content of the readers are equal or they both don't
+ * exist, false otherwise
+ * @throws NullPointerException if either input is null
+ * @throws IOException if an I/O error occurs
+ * @since 1.1
+ */
+ @SuppressWarnings("resource")
+ public static boolean contentEqualsOld(final Reader input1, final Reader input2)
+ throws IOException {
+ if (input1 == input2) {
+ return true;
+ }
+ if (input1 == null ^ input2 == null) {
+ return false;
+ }
+ final BufferedReader bufferedInput1 = toBufferedReader(input1);
+ final BufferedReader bufferedInput2 = toBufferedReader(input2);
+
+ int ch = bufferedInput1.read();
+ while (EOF != ch) {
+ final int ch2 = bufferedInput2.read();
+ if (ch != ch2) {
+ return false;
+ }
+ ch = bufferedInput1.read();
+ }
+
+ return bufferedInput2.read() == EOF;
+ }
+
+ public static String getString0() {
+ StringBuilder stringBuilder = new StringBuilder("ab");
+ for (int i = 0; i < 24; i++) {
Review comment:
Did you test with larger strings?
24 looks like a small string
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-753650101
Hello @XenoAmess
We already have a lot of changes for the next release, so I want to manage expectations such that I would I prefer to get out 3.12 before making even more big changes like these.
But still, let's continue this thread. Starting with the lowest-level bits: we need to justify the addition of the misnamed `Unsync*` classes, the prefix should be `Unsynchronized` like our existing `UnsynchronizedByteArrayInputStream`, which I've already mentioned. Perhaps our addition of UnsynchronizedByteArrayInputStream was a mistake since the Java folks make it sounds like these are superfluous in https://bugs.openjdk.java.net/browse/JDK-4097272, so I think we need to see a performance test that shows there is a clear performance benefit to adding those as valuable on their own.
We need performances test that show the differences, if any, between the JRE's classes and our proposed `Unsynchronized` versions. Since you propose two such classes `UnsyncBufferedInputStream` and `UnsyncBufferedReader`, that's two new tests. Or did I miss these here?
I think you should create a new PR for just these two new classes and their tests. This will make the work simpler for everyone when reviewing and testing.
TY.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-637035997
@garydgregory
Performance test done.
**Conclution:**
In cases for small content readers, the new function is slightly faster.
but in cases for large content, the new function can be 6.5 times faster.
```
D:\workspace\commons-io>mvn -Pbenchmark
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< commons-io:commons-io >------------------------
[INFO] Building Apache Commons IO 2.7.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ commons-io ---
[INFO] Deleting D:\workspace\commons-io\target
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[copy] Copying 2 files to D:\workspace\commons-io\target\apidocs\META-INF
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ commons-io ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ commons-io ---
[INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] Working directory: D:\workspace\commons-io
[INFO] Storing buildNumber: 73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 at timestamp: 1591034434847
[INFO] Storing buildScmBranch: refine_contentEquals
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\workspace\commons-io\src\main\resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\classes
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Recompile with -Xlint:deprecation for details.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOException
List.java uses unchecked or unsafe operations.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-bundle-plugin:4.2.1:manifest (bundle-manifest) @ commons-io ---
[INFO]
[INFO] --- animal-sniffer-maven-plugin:1.18:check (checkAPIcompatibility) @ commons-io ---
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java18:1.0
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:158: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:158: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:125: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.flip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:127: Covariant return type change detected: java.nio.Buffer java.
nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:207: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:209: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.flip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:213: Covariant return type change detected: java.nio.Buffer java.
nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:74: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:128: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:222: Covariant return type change detected: java.nio.Buffer
java.nio.CharBuffer.mark() has been changed to java.nio.CharBuffer java.nio.CharBuffer.mark()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:223: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.mark() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.mark()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:245: Covariant return type change detected: java.nio.Buffer
java.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:246: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.rewind() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:247: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:249: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.rewind() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:250: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:258: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.position(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:283: Covariant return type change detected: java.nio.Buffer jav
a.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:309: Covariant return type change detected: java.nio.Buffer jav
a.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:331: Covariant return type change detected: java.nio.Buffer jav
a.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:340: Covariant return type change detected: java.nio.Buffer jav
a.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:987: Covariant return type change detected: java.nio.Buffer java.nio.CharBuffer.f
lip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:2008: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffer.
position(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:2009: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffer.
limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] Copying 45 resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\test-classes
[INFO] /D:/workspace/commons-io/src/test/java/org/apache/commons/io/output/FileWriterWithEncodingTest.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/test/java/org/apache/commons/io/output/FileWriterWithEncodingTest.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (prepare-agent) @ commons-io ---
[INFO] argLine set to -javaagent:C:\\Users\\xenoa\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.5\\org.jacoco.agent-0.8.5-runtime.jar=destfile=D:\\workspa
ce\\commons-io\\target\\jacoco.exec
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ commons-io ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:exec (benchmark) @ commons-io ---
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/C:/Users/xenoa/.m2/repository/org/openjdk/jmh/jmh-core/1.21/jmh-core-1.21.jar) to field j
ava.io.PrintStream.charOut
WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew
# Run progress: 0.00% complete, ETA 00:33:20
# Fork: 1 of 5
# Warmup Iteration 1: 8403699.664 ns/op
# Warmup Iteration 2: 8342097.081 ns/op
# Warmup Iteration 3: 7431033.061 ns/op
# Warmup Iteration 4: 7714089.746 ns/op
# Warmup Iteration 5: 7338344.241 ns/op
Iteration 1: 7459707.308 ns/op
Iteration 2: 8792593.849 ns/op
Iteration 3: 10726891.104 ns/op
Iteration 4: 8550254.786 ns/op
Iteration 5: 8001410.312 ns/op
# Run progress: 5.00% complete, ETA 00:31:55
# Fork: 2 of 5
# Warmup Iteration 1: 7848885.804 ns/op
# Warmup Iteration 2: 7489775.075 ns/op
# Warmup Iteration 3: 7328874.652 ns/op
# Warmup Iteration 4: 7308097.663 ns/op
# Warmup Iteration 5: 7298969.001 ns/op
Iteration 1: 7347818.649 ns/op
Iteration 2: 7251497.464 ns/op
Iteration 3: 7366967.231 ns/op
Iteration 4: 7420473.516 ns/op
Iteration 5: 7517702.630 ns/op
# Run progress: 10.00% complete, ETA 00:30:14
# Fork: 3 of 5
# Warmup Iteration 1: 7403000.592 ns/op
# Warmup Iteration 2: 7237561.577 ns/op
# Warmup Iteration 3: 7328035.212 ns/op
# Warmup Iteration 4: 7310369.686 ns/op
# Warmup Iteration 5: 7258066.255 ns/op
Iteration 1: 7387172.526 ns/op
Iteration 2: 7278674.400 ns/op
Iteration 3: 7307894.887 ns/op
Iteration 4: 7363340.103 ns/op
Iteration 5: 7241780.811 ns/op
# Run progress: 15.00% complete, ETA 00:28:33
# Fork: 4 of 5
# Warmup Iteration 1: 7549794.264 ns/op
# Warmup Iteration 2: 7209186.671 ns/op
# Warmup Iteration 3: 7680527.859 ns/op
# Warmup Iteration 4: 7870276.336 ns/op
# Warmup Iteration 5: 8008019.376 ns/op
Iteration 1: 7868122.956 ns/op
Iteration 2: 8014526.603 ns/op
Iteration 3: 7610742.998 ns/op
Iteration 4: 7339908.804 ns/op
Iteration 5: 7466605.970 ns/op
# Run progress: 20.00% complete, ETA 00:26:52
# Fork: 5 of 5
# Warmup Iteration 1: 8062262.933 ns/op
# Warmup Iteration 2: 7888885.647 ns/op
# Warmup Iteration 3: 8092374.353 ns/op
# Warmup Iteration 4: 7535815.813 ns/op
# Warmup Iteration 5: 8376873.032 ns/op
Iteration 1: 7641931.603 ns/op
Iteration 2: 8134969.268 ns/op
Iteration 3: 8823474.603 ns/op
Iteration 4: 9548556.489 ns/op
Iteration 5: 8888150.622 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew":
7934046.780 ?99.9%) 644395.868 ns/op [Average]
(min, avg, max) = (7241780.811, 7934046.780, 10726891.104), stdev = 860250.047
CI (99.9%): [7289650.912, 8578442.648] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2
# Run progress: 25.00% complete, ETA 00:25:12
# Fork: 1 of 5
# Warmup Iteration 1: 427439412.500 ns/op
# Warmup Iteration 2: 411941108.000 ns/op
# Warmup Iteration 3: 415405856.000 ns/op
# Warmup Iteration 4: 408246064.000 ns/op
# Warmup Iteration 5: 381986007.407 ns/op
Iteration 1: 372559803.704 ns/op
Iteration 2: 372221244.444 ns/op
Iteration 3: 394561276.923 ns/op
Iteration 4: 410318660.000 ns/op
Iteration 5: 417270562.500 ns/op
# Run progress: 30.00% complete, ETA 00:23:36
# Fork: 2 of 5
# Warmup Iteration 1: 385519323.077 ns/op
# Warmup Iteration 2: 341417310.000 ns/op
# Warmup Iteration 3: 337620636.667 ns/op
# Warmup Iteration 4: 337741280.000 ns/op
# Warmup Iteration 5: 322947274.194 ns/op
Iteration 1: 334579670.000 ns/op
Iteration 2: 324313664.516 ns/op
Iteration 3: 349864203.448 ns/op
Iteration 4: 345000172.414 ns/op
Iteration 5: 338797430.000 ns/op
# Run progress: 35.00% complete, ETA 00:21:56
# Fork: 3 of 5
# Warmup Iteration 1: 424080325.000 ns/op
# Warmup Iteration 2: 409074288.000 ns/op
# Warmup Iteration 3: 397216476.923 ns/op
# Warmup Iteration 4: 379384881.481 ns/op
# Warmup Iteration 5: 406599224.000 ns/op
Iteration 1: 344006786.667 ns/op
Iteration 2: 318393962.500 ns/op
Iteration 3: 326383164.516 ns/op
Iteration 4: 326728512.903 ns/op
Iteration 5: 339791536.667 ns/op
# Run progress: 40.00% complete, ETA 00:20:17
# Fork: 4 of 5
# Warmup Iteration 1: 359791750.000 ns/op
# Warmup Iteration 2: 344011470.000 ns/op
# Warmup Iteration 3: 317443868.750 ns/op
# Warmup Iteration 4: 366668921.429 ns/op
# Warmup Iteration 5: 340980226.667 ns/op
Iteration 1: 362224464.286 ns/op
Iteration 2: 329631980.645 ns/op
Iteration 3: 328959990.323 ns/op
Iteration 4: 331093680.645 ns/op
Iteration 5: 328042438.710 ns/op
# Run progress: 45.00% complete, ETA 00:18:37
# Fork: 5 of 5
# Warmup Iteration 1: 328533070.968 ns/op
# Warmup Iteration 2: 321877478.125 ns/op
# Warmup Iteration 3: 315723571.875 ns/op
# Warmup Iteration 4: 323838003.226 ns/op
# Warmup Iteration 5: 325582848.387 ns/op
Iteration 1: 322872964.516 ns/op
Iteration 2: 320972165.625 ns/op
Iteration 3: 322146753.125 ns/op
Iteration 4: 320983556.250 ns/op
Iteration 5: 332471216.129 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2":
344567594.458 ?99.9%) 21104733.591 ns/op [Average]
(min, avg, max) = (318393962.500, 344567594.458, 417270562.500), stdev = 28174215.532
CI (99.9%): [323462860.868, 365672328.049] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld
# Run progress: 50.00% complete, ETA 00:16:57
# Fork: 1 of 5
# Warmup Iteration 1: 8040772.910 ns/op
# Warmup Iteration 2: 7850546.468 ns/op
# Warmup Iteration 3: 8366507.191 ns/op
# Warmup Iteration 4: 8919272.193 ns/op
# Warmup Iteration 5: 8934492.411 ns/op
Iteration 1: 9849140.059 ns/op
Iteration 2: 10280016.496 ns/op
Iteration 3: 9377572.540 ns/op
Iteration 4: 8634857.550 ns/op
Iteration 5: 8796502.111 ns/op
# Run progress: 55.00% complete, ETA 00:15:14
# Fork: 2 of 5
# Warmup Iteration 1: 9196890.441 ns/op
# Warmup Iteration 2: 8651530.769 ns/op
# Warmup Iteration 3: 9130613.504 ns/op
# Warmup Iteration 4: 9528571.075 ns/op
# Warmup Iteration 5: 8603469.046 ns/op
Iteration 1: 8649056.612 ns/op
Iteration 2: 8740562.707 ns/op
Iteration 3: 8636735.807 ns/op
Iteration 4: 8658414.545 ns/op
Iteration 5: 9077745.554 ns/op
# Run progress: 60.00% complete, ETA 00:13:32
# Fork: 3 of 5
# Warmup Iteration 1: 9210640.074 ns/op
# Warmup Iteration 2: 7950668.124 ns/op
# Warmup Iteration 3: 8049352.132 ns/op
# Warmup Iteration 4: 7906520.949 ns/op
# Warmup Iteration 5: 8193835.545 ns/op
Iteration 1: 7833574.706 ns/op
Iteration 2: 8025837.640 ns/op
Iteration 3: 8621776.034 ns/op
Iteration 4: 8516147.915 ns/op
Iteration 5: 8379908.710 ns/op
# Run progress: 65.00% complete, ETA 00:11:50
# Fork: 4 of 5
# Warmup Iteration 1: 8810415.141 ns/op
# Warmup Iteration 2: 8794569.859 ns/op
# Warmup Iteration 3: 8730751.745 ns/op
# Warmup Iteration 4: 8563348.202 ns/op
# Warmup Iteration 5: 8946487.757 ns/op
Iteration 1: 8798953.725 ns/op
Iteration 2: 9077297.461 ns/op
Iteration 3: 9203544.250 ns/op
Iteration 4: 10801144.492 ns/op
Iteration 5: 10137045.694 ns/op
# Run progress: 70.00% complete, ETA 00:10:09
# Fork: 5 of 5
# Warmup Iteration 1: 10034710.933 ns/op
# Warmup Iteration 2: 7852066.954 ns/op
# Warmup Iteration 3: 8036246.265 ns/op
# Warmup Iteration 4: 8208804.184 ns/op
# Warmup Iteration 5: 8919215.241 ns/op
Iteration 1: 7703625.019 ns/op
Iteration 2: 8173784.069 ns/op
Iteration 3: 8476930.000 ns/op
Iteration 4: 9254641.073 ns/op
Iteration 5: 8732920.419 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld":
8897509.408 ?99.9%) 558227.023 ns/op [Average]
(min, avg, max) = (7703625.019, 8897509.408, 10801144.492), stdev = 745217.105
CI (99.9%): [8339282.385, 9455736.431] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2
# Run progress: 75.00% complete, ETA 00:08:27
# Fork: 1 of 5
# Warmup Iteration 1: 2274597420.000 ns/op
# Warmup Iteration 2: 2161414140.000 ns/op
# Warmup Iteration 3: 2060648900.000 ns/op
# Warmup Iteration 4: 2034553880.000 ns/op
# Warmup Iteration 5: 2036716060.000 ns/op
Iteration 1: 2039811500.000 ns/op
Iteration 2: 2029332780.000 ns/op
Iteration 3: 2048828760.000 ns/op
Iteration 4: 2167898680.000 ns/op
Iteration 5: 2375598240.000 ns/op
# Run progress: 80.00% complete, ETA 00:06:47
# Fork: 2 of 5
# Warmup Iteration 1: 2302730460.000 ns/op
# Warmup Iteration 2: 2169550200.000 ns/op
# Warmup Iteration 3: 2183069220.000 ns/op
# Warmup Iteration 4: 2219912760.000 ns/op
# Warmup Iteration 5: 2276130160.000 ns/op
Iteration 1: 2256782140.000 ns/op
Iteration 2: 2212519940.000 ns/op
Iteration 3: 2194957540.000 ns/op
Iteration 4: 2139474860.000 ns/op
Iteration 5: 2239168680.000 ns/op
# Run progress: 85.00% complete, ETA 00:05:07
# Fork: 3 of 5
# Warmup Iteration 1: 2871005625.000 ns/op
# Warmup Iteration 2: 2234621780.000 ns/op
# Warmup Iteration 3: 2057113240.000 ns/op
# Warmup Iteration 4: 2106691240.000 ns/op
# Warmup Iteration 5: 2239427440.000 ns/op
Iteration 1: 2221194720.000 ns/op
Iteration 2: 2252754040.000 ns/op
Iteration 3: 2362948520.000 ns/op
Iteration 4: 2233297260.000 ns/op
Iteration 5: 2289872180.000 ns/op
# Run progress: 90.00% complete, ETA 00:03:25
# Fork: 4 of 5
# Warmup Iteration 1: 2316297000.000 ns/op
# Warmup Iteration 2: 2186447840.000 ns/op
# Warmup Iteration 3: 2186496100.000 ns/op
# Warmup Iteration 4: 2210486500.000 ns/op
# Warmup Iteration 5: 2248586640.000 ns/op
Iteration 1: 2228412480.000 ns/op
Iteration 2: 2293236620.000 ns/op
Iteration 3: 2300933160.000 ns/op
Iteration 4: 2050526440.000 ns/op
Iteration 5: 2103876620.000 ns/op
# Run progress: 95.00% complete, ETA 00:01:43
# Fork: 5 of 5
# Warmup Iteration 1: 2231988940.000 ns/op
# Warmup Iteration 2: 2132075180.000 ns/op
# Warmup Iteration 3: 2107424340.000 ns/op
# Warmup Iteration 4: 2244625860.000 ns/op
# Warmup Iteration 5: 2212587240.000 ns/op
Iteration 1: 2358689240.000 ns/op
Iteration 2: 2360920240.000 ns/op
Iteration 3: 2304193360.000 ns/op
Iteration 4: 2120541520.000 ns/op
Iteration 5: 2225049120.000 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2":
2216432745.600 ?99.9%) 79521556.878 ns/op [Average]
(min, avg, max) = (2029332780.000, 2216432745.600, 2375598240.000), stdev = 106159003.301
CI (99.9%): [2136911188.722, 2295954302.478] (assumes normal distribution)
# Run complete. Total time: 00:34:37
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew avgt 25 7934046.780 ? 644395.868 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2 avgt 25 344567594.458 ?21104733.591 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsOld avgt 25 8897509.408 ? 558227.023 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2 avgt 25 2216432745.600 ?79521556.878 ns/op
Benchmark result is saved to target/jmh-result.org.apache.json
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.8.2:attach-descriptor (attach-descriptor) @ commons-io ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:test-jar (default) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.0:jar-no-fork (create-source-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-sources.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.0:test-jar-no-fork (create-source-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-test-sources.jar
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:check (check) @ commons-io ---
[INFO] Skipping JaCoCo execution due to missing execution data file:D:\workspace\commons-io\target\jacoco.exec
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (default-cli) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] >>> clirr-maven-plugin:2.8:check (default-cli) > compile @ commons-io >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ commons-io ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ commons-io ---
[INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] Working directory: D:\workspace\commons-io
[INFO] Storing buildNumber: 73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 at timestamp: 1591036535724
[INFO] Storing buildScmBranch: refine_contentEquals
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\workspace\commons-io\src\main\resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\classes
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Recompile with -Xlint:deprecation for details.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOException
List.java uses unchecked or unsafe operations.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] <<< clirr-maven-plugin:2.8:check (default-cli) < compile @ commons-io <<<
[INFO]
[INFO]
[INFO] --- clirr-maven-plugin:2.8:check (default-cli) @ commons-io ---
[INFO] Comparing to version: 2.7
[INFO] Succeeded with 0 errors; 0 warnings; and 0 other changes.
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.0:check (default-cli) @ commons-io ---
[INFO]
[INFO] >>> maven-javadoc-plugin:3.1.1:javadoc (default-cli) > generate-sources @ commons-io >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] <<< maven-javadoc-plugin:3.1.1:javadoc (default-cli) < generate-sources @ commons-io <<<
[INFO]
[INFO]
[INFO] --- maven-javadoc-plugin:3.1.1:javadoc (default-cli) @ commons-io ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 35:31 min
[INFO] Finished at: 2020-06-02T02:35:59+08:00
[INFO] ------------------------------------------------------------------------
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-668359292
@garydgregory
got it.
new added class renamed as:
org.apache.commons.io.input.buffer.UnsyncBufferedInputStream
org.apache.commons.io.input.buffer.UnsyncBufferedReader
org.apache.commons.io.input.buffer.LineEndUnifiedBufferedReader
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465613818
##########
File path: src/main/java/org/apache/commons/io/input/buffer/UnsyncBufferedInputStream.java
##########
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.input.buffer;
+
+import java.io.IOException;
+import java.io.InputStream;
+import org.apache.commons.io.IOUtils;
+import static org.apache.commons.io.IOUtils.EOF;
+
+/**
+ * A BufferedReader class who does not care about thread safety, but very much faster.
+ *
+ * Should be able to replace java.io.BufferedReader in nearly every use-cases when you
+ * need the Reader be buffered, but do not need it have thread safety.
+ */
+public class UnsyncBufferedInputStream extends InputStream {
+ protected final InputStream inputStream;
+ protected final byte[] byteBuffer;
+
+ protected int nowIndex = 0;
+ protected int nowLimit = 0;
+
Review comment:
Fields should be private, otherwise they become part of the API and cannot be changed easily.
They also make thorough testing much harder.
If necessary, getters and/or setters can be added.
But only if necessary, as they increase the testing requirements, and cannot be dropped once added.
Similarly elsewhere.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-668333108
On Mon, Aug 3, 2020 at 7:42 PM sebbASF <no...@github.com> wrote:
> The original comparison methods are easy to follow.
> It would be good to keep the same clarity by defining new versions of
> BufferedReader that have good single-threaded performance. Such classes
> would also be useful elsewhere.
> So for example
> BufferedReader bufferedInput1 = toBufferedReader(input1);
> would become something like:
> FastBufferedReader bufferedInput1 = toFastBuffer(input1) (*)
>
> where toFastBuffer() works like buffer() but returns an instance of a
> non-threadsafe FastBufferedReader.
>
> The FastBufferedxxx classes would also be useful in their own right
> elsewhere in IO.
> Obviously the documentation would need to make it clear that they are not
> thread-safe.
> I suspect they don't need to implement mark (which would make them
> simpler).
>
> (*) FastBuffer is just a suggestion for the name. Maybe QandDBuffer would
> be better (Quick and Dirty)!
>
I would really like to stay away from names like "Fast". Faster than what?
What if we come up with something better but incompatible, would that be a
"FasterThanFast" or "Fastest" ;-)
We've run into this kind naming issue before and we decide to go with a
prefix that describes the actual functionality which is "unsynchronized",
see
- org.apache.commons.io.input.UnsynchronizedByteArrayInputStream
- org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream
Note that there is precedent for this kind of name in the JRE's copy of
Apache's XML code:
- com.sun.org.apache.xml.internal.security.utils.UnsyncBufferedOutputStream
- com.sun.org.apache.xml.internal.security.utils.UnsyncByteArrayOutputStream
Gary
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/commons-io/pull/118#issuecomment-668293267>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJB6N7FXXKQAFHZ5JBELY3R65DPFANCNFSM4NP6TZTQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669097163
AFAICT the new classes UnsyncBufferedReader etc are not actually used (except in test code).
I would expect all the contentEquals methods to remain exactly the same except for changes to the input sources.
The logic should remain unchanged (otherwise the new classes are not proper replacements).
Likewise I would not expect to see any changes to test cases, only new ones for the new classes, and perhaps a few new samples for existing methods where the coverage is incomplete.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636989562
@XenoAmess
- When you create a PR, please do not leave the description empty. It makes it harder to review.
- If a PR claims a performance improvement, it must back it up; some components do this using JMH microbenchmarks, for example please see Apache Commons BCEL, Crypto, CSV, Lang, Math, RNG.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636994263
@garydgregory
> @XenoAmess
>
> * When you create a PR, please do not leave the description empty. It makes it harder to review.
OK
> * If a PR claims a performance improvement, it must back it up; some components do this using JMH microbenchmarks, for example please see Apache Commons BCEL, Crypto, CSV, Lang, Math, RNG.
I have only experiences using Jprofiler and I don't know how to embed a test with performance in junit.
I'll learn about JMH microbenchmarks.
thanks.
update: I learned how to use it, thanks for your recommendation.
also, need I create more tests for testing wether this function is correct?
or tests that already exist ,which for the older function, is good enough and I need not create more?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-637036540
result json:
```
[
{
"jmhVersion" : "1.21",
"benchmark" : "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew",
"mode" : "avgt",
"threads" : 1,
"forks" : 5,
"jvm" : "C:\\jdk-13.0.2+8\\bin\\java.exe",
"jvmArgs" : [
],
"jdkVersion" : "13.0.2",
"vmName" : "OpenJDK 64-Bit Server VM",
"vmVersion" : "13.0.2+8",
"warmupIterations" : 5,
"warmupTime" : "10 s",
"warmupBatchSize" : 1,
"measurementIterations" : 5,
"measurementTime" : "10 s",
"measurementBatchSize" : 1,
"primaryMetric" : {
"score" : 7934046.779660092,
"scoreError" : 644395.8679898068,
"scoreConfidence" : [
7289650.911670285,
8578442.6476499
],
"scorePercentiles" : {
"0.0" : 7241780.811006517,
"50.0" : 7517702.629601804,
"90.0" : 9152312.968421623,
"95.0" : 1.0373390719340876E7,
"99.0" : 1.0726891103965702E7,
"99.9" : 1.0726891103965702E7,
"99.99" : 1.0726891103965702E7,
"99.999" : 1.0726891103965702E7,
"99.9999" : 1.0726891103965702E7,
"100.0" : 1.0726891103965702E7
},
"scoreUnit" : "ns/op",
"rawData" : [
[
7459707.30797912,
8792593.848857645,
1.0726891103965702E7,
8550254.786324786,
8001410.311750599
],
[
7347818.649045521,
7251497.463768116,
7366967.231222386,
7420473.516320474,
7517702.629601804
],
[
7387172.525849335,
7278674.4,
7307894.88677867,
7363340.103016924,
7241780.811006517
],
[
7868122.955974842,
8014526.602564103,
7610742.99847793,
7339908.804108584,
7466605.9701492535
],
[
7641931.603053435,
8134969.268292683,
8823474.603174603,
9548556.488549618,
8888150.621669628
]
]
},
"secondaryMetrics" : {
}
},
{
"jmhVersion" : "1.21",
"benchmark" : "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2",
"mode" : "avgt",
"threads" : 1,
"forks" : 5,
"jvm" : "C:\\jdk-13.0.2+8\\bin\\java.exe",
"jvmArgs" : [
],
"jdkVersion" : "13.0.2",
"vmName" : "OpenJDK 64-Bit Server VM",
"vmVersion" : "13.0.2+8",
"warmupIterations" : 5,
"warmupTime" : "10 s",
"warmupBatchSize" : 1,
"measurementIterations" : 5,
"measurementTime" : "10 s",
"measurementBatchSize" : 1,
"primaryMetric" : {
"score" : 3.4456759445822275E8,
"scoreError" : 2.110473359059532E7,
"scoreConfidence" : [
3.2346286086762744E8,
3.6567232804881805E8
],
"scorePercentiles" : {
"0.0" : 3.183939625E8,
"50.0" : 3.3247121612903225E8,
"90.0" : 4.0086423015384614E8,
"95.0" : 4.1518499175E8,
"99.0" : 4.172705625E8,
"99.9" : 4.172705625E8,
"99.99" : 4.172705625E8,
"99.999" : 4.172705625E8,
"99.9999" : 4.172705625E8,
"100.0" : 4.172705625E8
},
"scoreUnit" : "ns/op",
"rawData" : [
[
3.725598037037037E8,
3.722212444444444E8,
3.945612769230769E8,
4.1031866E8,
4.172705625E8
],
[
3.3457967E8,
3.24313664516129E8,
3.4986420344827586E8,
3.450001724137931E8,
3.3879743E8
],
[
3.440067866666667E8,
3.183939625E8,
3.26383164516129E8,
3.267285129032258E8,
3.397915366666667E8
],
[
3.6222446428571427E8,
3.296319806451613E8,
3.2895999032258064E8,
3.310936806451613E8,
3.280424387096774E8
],
[
3.22872964516129E8,
3.20972165625E8,
3.22146753125E8,
3.2098355625E8,
3.3247121612903225E8
]
]
},
"secondaryMetrics" : {
}
},
{
"jmhVersion" : "1.21",
"benchmark" : "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld",
"mode" : "avgt",
"threads" : 1,
"forks" : 5,
"jvm" : "C:\\jdk-13.0.2+8\\bin\\java.exe",
"jvmArgs" : [
],
"jdkVersion" : "13.0.2",
"vmName" : "OpenJDK 64-Bit Server VM",
"vmVersion" : "13.0.2+8",
"warmupIterations" : 5,
"warmupTime" : "10 s",
"warmupBatchSize" : 1,
"measurementIterations" : 5,
"measurementTime" : "10 s",
"measurementBatchSize" : 1,
"primaryMetric" : {
"score" : 8897509.407577794,
"scoreError" : 558227.0229691897,
"scoreConfidence" : [
8339282.384608604,
9455736.430546984
],
"scorePercentiles" : {
"0.0" : 7703625.019245573,
"50.0" : 8732920.418848168,
"90.0" : 1.019423401477403E7,
"95.0" : 1.0644806093478914E7,
"99.0" : 1.0801144492440606E7,
"99.9" : 1.0801144492440606E7,
"99.99" : 1.0801144492440606E7,
"99.999" : 1.0801144492440606E7,
"99.9999" : 1.0801144492440606E7,
"100.0" : 1.0801144492440606E7
},
"scoreUnit" : "ns/op",
"rawData" : [
[
9849140.059055118,
1.0280016495901639E7,
9377572.539831303,
8634857.549611734,
8796502.110817943
],
[
8649056.611927398,
8740562.70742358,
8636735.806729939,
8658414.545454545,
9077745.55353902
],
[
7833574.706342991,
8025837.640449438,
8621776.034482758,
8516147.914893618,
8379908.710217755
],
[
8798953.724802805,
9077297.46146872,
9203544.25022999,
1.0801144492440606E7,
1.013704569402229E7
],
[
7703625.019245573,
8173784.068627451,
8476930.0,
9254641.073080482,
8732920.418848168
]
]
},
"secondaryMetrics" : {
}
},
{
"jmhVersion" : "1.21",
"benchmark" : "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2",
"mode" : "avgt",
"threads" : 1,
"forks" : 5,
"jvm" : "C:\\jdk-13.0.2+8\\bin\\java.exe",
"jvmArgs" : [
],
"jdkVersion" : "13.0.2",
"vmName" : "OpenJDK 64-Bit Server VM",
"vmVersion" : "13.0.2+8",
"warmupIterations" : 5,
"warmupTime" : "10 s",
"warmupBatchSize" : 1,
"measurementIterations" : 5,
"measurementTime" : "10 s",
"measurementBatchSize" : 1,
"primaryMetric" : {
"score" : 2.2164327456E9,
"scoreError" : 7.952155687779064E7,
"scoreConfidence" : [
2.1369111887222092E9,
2.2959543024777904E9
],
"scorePercentiles" : {
"0.0" : 2.02933278E9,
"50.0" : 2.22841248E9,
"90.0" : 2.361731552E9,
"95.0" : 2.371803324E9,
"99.0" : 2.37559824E9,
"99.9" : 2.37559824E9,
"99.99" : 2.37559824E9,
"99.999" : 2.37559824E9,
"99.9999" : 2.37559824E9,
"100.0" : 2.37559824E9
},
"scoreUnit" : "ns/op",
"rawData" : [
[
2.0398115E9,
2.02933278E9,
2.04882876E9,
2.16789868E9,
2.37559824E9
],
[
2.25678214E9,
2.21251994E9,
2.19495754E9,
2.13947486E9,
2.23916868E9
],
[
2.22119472E9,
2.25275404E9,
2.36294852E9,
2.23329726E9,
2.28987218E9
],
[
2.22841248E9,
2.29323662E9,
2.30093316E9,
2.05052644E9,
2.10387662E9
],
[
2.35868924E9,
2.36092024E9,
2.30419336E9,
2.12054152E9,
2.22504912E9
]
]
},
"secondaryMetrics" : {
}
}
]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32434479)
Coverage decreased (-1.4%) to 88.339% when pulling **3ab7e0814d1968e5b5ee37a87a626ea2aab00b59 on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462150050
##########
File path: src/test/java/org/apache/commons/io/performance/IOUtilsContentEqualsPerformanceTest.java
##########
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.performance;
+
+import org.apache.commons.io.IOUtils;
+import org.openjdk.jmh.annotations.*;
+
+import java.io.*;
+import java.util.concurrent.TimeUnit;
+
+import static org.apache.commons.io.IOUtils.EOF;
+import static org.apache.commons.io.IOUtils.toBufferedReader;
+
+/**
+ * Test to show whether using BitSet for removeAll() methods is faster than using HashSet.
+ */
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+@State(Scope.Thread)
+public class IOUtilsContentEqualsPerformanceTest {
+
+ private static final String[] STRINGS = new String[3];
+
+ static {
+ STRINGS[0] = getString0();
+ STRINGS[1] = STRINGS[0] + 'c';
+ STRINGS[2] = STRINGS[0] + 'd';
+ }
+
+ private static final int LOOP = 10;
+
+ @Benchmark
+ public boolean[] testContentEqualsForFileNew() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsNew2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ /**
+ * Old version of IOUtils.contentEquals(Reader, Reader)
+ *
+ * Compares the contents of two Readers to determine if they are equal or
+ * not.
+ * <p>
+ * This method buffers the input internally using
+ * <code>BufferedReader</code> if they are not already buffered.
+ * </p>
+ *
+ * @param input1 the first reader
+ * @param input2 the second reader
+ * @return true if the content of the readers are equal or they both don't
+ * exist, false otherwise
+ * @throws NullPointerException if either input is null
+ * @throws IOException if an I/O error occurs
+ * @since 1.1
+ */
+ @SuppressWarnings("resource")
+ public static boolean contentEqualsOld(final Reader input1, final Reader input2)
+ throws IOException {
+ if (input1 == input2) {
+ return true;
+ }
+ if (input1 == null ^ input2 == null) {
+ return false;
+ }
+ final BufferedReader bufferedInput1 = toBufferedReader(input1);
+ final BufferedReader bufferedInput2 = toBufferedReader(input2);
+
+ int ch = bufferedInput1.read();
+ while (EOF != ch) {
+ final int ch2 = bufferedInput2.read();
+ if (ch != ch2) {
+ return false;
+ }
+ ch = bufferedInput1.read();
+ }
+
+ return bufferedInput2.read() == EOF;
+ }
+
+ public static String getString0() {
+ StringBuilder stringBuilder = new StringBuilder("ab");
+ for (int i = 0; i < 24; i++) {
Review comment:
@eolivelli
stringBuilder.append(stringBuilder);
that is 2^24, not 2*24...
And I think that is long enough.
If I use lang3.StringUtils.repeat maybe it will be more clear, but I'm not quite sure whethere it be good to use it in commons-io.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -752,19 +777,46 @@ public static boolean contentEquals(final Reader input1, final Reader input2)
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader bufferedInput1 = toBufferedReader(input1);
- final BufferedReader bufferedInput2 = toBufferedReader(input2);
- int ch = bufferedInput1.read();
- while (EOF != ch) {
- final int ch2 = bufferedInput2.read();
- if (ch != ch2) {
- return false;
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1;
+ int nowPos2;
+ int nowRead1;
+ int nowRead2;
+ while (true) {
+ nowPos1 = 0;
+ nowPos2 = 0;
+ for (int nowCheck = 0; nowCheck < CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE; nowCheck++) {
+ if (nowPos1 == nowCheck) {
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ return nowPos2 == nowCheck && input2.read() == -1;
+ }
+ nowPos1 += nowRead1;
+ }
+ if (nowPos2 == nowCheck) {
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ return nowPos1 == nowCheck && input1.read() == -1;
+ }
+ nowPos2 += nowRead2;
+ }
+ if (charArray1[nowCheck] != charArray2[nowCheck]) {
+ return false;
+ }
}
- ch = bufferedInput1.read();
}
+ }
- return bufferedInput2.read() == EOF;
+ private enum LastState {
+ r,
Review comment:
> Can you please use UPPERCASE identifiers?
Also, IMHO it is better to give meaningful names or add a minimal explanation
@eolivelli
OK, will do.
##########
File path: src/test/java/org/apache/commons/io/performance/IOUtilsContentEqualsPerformanceTest.java
##########
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.io.performance;
+
+import org.apache.commons.io.IOUtils;
+import org.openjdk.jmh.annotations.*;
+
+import java.io.*;
+import java.util.concurrent.TimeUnit;
+
+import static org.apache.commons.io.IOUtils.EOF;
+import static org.apache.commons.io.IOUtils.toBufferedReader;
+
+/**
+ * Test to show whether using BitSet for removeAll() methods is faster than using HashSet.
+ */
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+@State(Scope.Thread)
+public class IOUtilsContentEqualsPerformanceTest {
+
+ private static final String[] STRINGS = new String[3];
+
+ static {
+ STRINGS[0] = getString0();
+ STRINGS[1] = STRINGS[0] + 'c';
+ STRINGS[2] = STRINGS[0] + 'd';
+ }
+
+ private static final int LOOP = 10;
+
+ @Benchmark
+ public boolean[] testContentEqualsForFileNew() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld() throws IOException {
+ boolean[] res = new boolean[2];
+ for (int i = 0; i < LOOP; i++) {
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileNoBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[0] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ try (InputStream inputStream1 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ InputStream inputStream2 =
+ this.getClass().getResourceAsStream("/org/apache/commons/io/testfileBOM.xml");
+ Reader inputReader1 = new InputStreamReader(inputStream1);
+ Reader inputReader2 = new InputStreamReader(inputStream2);
+ ) {
+ res[1] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsNew2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = IOUtils.contentEquals(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ @Benchmark
+ public boolean[] testContentEqualsOld2() throws IOException {
+ boolean[] res = new boolean[9];
+ for (int i = 0; i < 3; i++) {
+ for (int j = 0; j < 3; j++) {
+ try (Reader inputReader1 = new StringReader(STRINGS[i]);
+ Reader inputReader2 = new StringReader(STRINGS[j]);
+ ) {
+ res[i * 3 + j] = contentEqualsOld(inputReader1, inputReader2);
+ }
+ }
+ }
+ return res;
+ }
+
+ /**
+ * Old version of IOUtils.contentEquals(Reader, Reader)
+ *
+ * Compares the contents of two Readers to determine if they are equal or
+ * not.
+ * <p>
+ * This method buffers the input internally using
+ * <code>BufferedReader</code> if they are not already buffered.
+ * </p>
+ *
+ * @param input1 the first reader
+ * @param input2 the second reader
+ * @return true if the content of the readers are equal or they both don't
+ * exist, false otherwise
+ * @throws NullPointerException if either input is null
+ * @throws IOException if an I/O error occurs
+ * @since 1.1
+ */
+ @SuppressWarnings("resource")
+ public static boolean contentEqualsOld(final Reader input1, final Reader input2)
+ throws IOException {
+ if (input1 == input2) {
+ return true;
+ }
+ if (input1 == null ^ input2 == null) {
+ return false;
+ }
+ final BufferedReader bufferedInput1 = toBufferedReader(input1);
+ final BufferedReader bufferedInput2 = toBufferedReader(input2);
+
+ int ch = bufferedInput1.read();
+ while (EOF != ch) {
+ final int ch2 = bufferedInput2.read();
+ if (ch != ch2) {
+ return false;
+ }
+ ch = bufferedInput1.read();
+ }
+
+ return bufferedInput2.read() == EOF;
+ }
+
+ public static String getString0() {
+ StringBuilder stringBuilder = new StringBuilder("ab");
+ for (int i = 0; i < 24; i++) {
Review comment:
> Did you test with larger strings?
24 looks like a small string
@eolivelli
stringBuilder.append(stringBuilder);
that is 2^24, not 2*24...
And I think that is long enough.
If I use lang3.StringUtils.repeat maybe it will be more clear, but I'm not quite sure whethere it be good to use it in commons-io.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -790,16 +842,342 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader br1 = toBufferedReader(input1);
- final BufferedReader br2 = toBufferedReader(input2);
- String line1 = br1.readLine();
- String line2 = br2.readLine();
- while (line1 != null && line1.equals(line2)) {
- line1 = br1.readLine();
- line2 = br2.readLine();
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1 = 0;
+ int nowPos2 = 0;
+ int nowRead1;
+ int nowRead2;
+ int nowCheck1 = 0;
+ int nowCheck2 = 0;
+ boolean readEnd1 = false;
+ boolean readEnd2 = false;
+ LastState lastState1 = LastState.newLine;
+ LastState lastState2 = LastState.newLine;
+ while (true) {
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd1) {
+ if (readEnd2) {
+ return true;
+ } else {
+ switch (lastState1) {
+ case r:
+ case newLine:
+ switch (lastState2) {
+ case r:
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState2) {
+ case normal:
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd2) {
+ return true;
+ } else if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ case '\n':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+ } else if (readEnd2) {
+ switch (lastState2) {
+ case r:
+ case newLine:
+ switch (lastState1) {
+ case r:
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState1) {
+ case normal:
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (readEnd1) {
+ return true;
+ } else if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ lastState1 = lastState2 = LastState.r;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck2++;
+ if (readEnd1) {
+ continue;
+ }
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ }
+ continue;
+ default:
+ return false;
+ }
+ case '\n':
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ if (readEnd2) {
+ continue;
+ }
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ }
+ continue;
+ default:
+ if (lastState1 == LastState.r) {
+ lastState1 = LastState.newLine;
+ nowCheck1++;
+ continue;
+ } else {
+ return false;
+ }
+ }
+ default:
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ if (lastState2 == LastState.r) {
+ lastState2 = LastState.newLine;
+ nowCheck2++;
+ continue;
+ } else {
+ return false;
+ }
+ case '\r':
+ return false;
+ default:
+ if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
+ return false;
+ }
+ lastState1 = lastState2 = LastState.normal;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ }
+ }
}
Review comment:
@garydgregory @melloware
> there are zero comments in hundreds of lines of code
comments just added now:)
please see the latest pr.
> In some cases performance gains do not outweigh complexity of the code.
Yes.
It is also out of my thought.
I thought it SHOULD be also a 500%+ faster like the contentEquals function, but it is actually only 92% faster in jmh.
So it might not worth that much inline.
I will try to split some sub-functions out later today, and hope that can reduce the size of this function.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -790,16 +842,342 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader br1 = toBufferedReader(input1);
- final BufferedReader br2 = toBufferedReader(input2);
- String line1 = br1.readLine();
- String line2 = br2.readLine();
- while (line1 != null && line1.equals(line2)) {
- line1 = br1.readLine();
- line2 = br2.readLine();
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1 = 0;
+ int nowPos2 = 0;
+ int nowRead1;
+ int nowRead2;
+ int nowCheck1 = 0;
+ int nowCheck2 = 0;
+ boolean readEnd1 = false;
+ boolean readEnd2 = false;
+ LastState lastState1 = LastState.newLine;
+ LastState lastState2 = LastState.newLine;
+ while (true) {
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd1) {
+ if (readEnd2) {
+ return true;
+ } else {
+ switch (lastState1) {
+ case r:
+ case newLine:
+ switch (lastState2) {
+ case r:
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState2) {
+ case normal:
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd2) {
+ return true;
+ } else if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ case '\n':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+ } else if (readEnd2) {
+ switch (lastState2) {
+ case r:
+ case newLine:
+ switch (lastState1) {
+ case r:
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState1) {
+ case normal:
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (readEnd1) {
+ return true;
+ } else if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ lastState1 = lastState2 = LastState.r;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck2++;
+ if (readEnd1) {
+ continue;
+ }
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ }
+ continue;
+ default:
+ return false;
+ }
+ case '\n':
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ if (readEnd2) {
+ continue;
+ }
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ }
+ continue;
+ default:
+ if (lastState1 == LastState.r) {
+ lastState1 = LastState.newLine;
+ nowCheck1++;
+ continue;
+ } else {
+ return false;
+ }
+ }
+ default:
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ if (lastState2 == LastState.r) {
+ lastState2 = LastState.newLine;
+ nowCheck2++;
+ continue;
+ } else {
+ return false;
+ }
+ case '\r':
+ return false;
+ default:
+ if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
+ return false;
+ }
+ lastState1 = lastState2 = LastState.normal;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ }
+ }
}
Review comment:
@garydgregory @melloware
> there are zero comments in hundreds of lines of code
comments just added now:)
please see the latest pr.
> In some cases performance gains do not outweigh complexity of the code.
Yes.
It is also out of my thought.
I thought it SHOULD be also a 500%+ faster like the contentEquals function, but it is actually only 92% faster in jmh.
So it might not worth that much inline.
I will try to split some sub-functions out later today, and hope that can reduce the size of this function.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -1147,31 +1201,43 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
}
continue;
default:
- if (lastState1 == LastState.r) {
- lastState1 = LastState.newLine;
+ // if input2's next is normal.
+ // if input1's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState1 == LastState.R) {
+ lastState1 = LastState.NEW_LINE;
nowCheck1++;
continue;
} else {
return false;
}
}
default:
+ // if input1's next is normal.
switch (charArray2[nowCheck2]) {
case '\n':
- if (lastState2 == LastState.r) {
- lastState2 = LastState.newLine;
+ // if input2's next is '\n'.
+ // if input2's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState2 == LastState.R) {
+ lastState2 = LastState.NEW_LINE;
nowCheck2++;
continue;
} else {
return false;
}
case '\r':
+ // if input2's next is '\r'.
+ // illegal.
return false;
default:
+ // if input2's next is normal.
+ // if equal then legal.
+ // otherwise illegal.
if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
return false;
}
- lastState1 = lastState2 = LastState.normal;
+ lastState1 = lastState2 = LastState.NORMAL;
nowCheck1++;
nowCheck2++;
continue;
Review comment:
> Most of the comments above are unnecessary if the variables are well named and commented.
> What is missing is how the algorithm works, and what the various combinations of state actually mean in terms of the algorithm.
>
> I'm not clear why the code sometimes checks for '\r' and sometimes uses the enum.
> Why not use a variable containing the last character?
>
> ==
>
> I suspect the code could be much simplified by using a suitable filter on the inputs.
> There are several examples in IO, and NET has FromNetASCIIInputStream which deals with CRLF conversions
> It occurs to me that the code is effectively buffering the output from a BufferedReader (or BufferedInputStream).
> One would expect these classes to be reasonably fast, however the JVM has to do locking and other checks in order to support multi-threading. It has to do this for each read() call.
>
> Rather than implement the buffering directly in the compare methods, it might be better to implement a generic, non-threadsafe filter that can be used in situations such as these.
>
> The original code should then work without any change other than to add the filter.
> It occurs to me that the code is effectively buffering the output from a BufferedReader (or BufferedInputStream).
> One would expect these classes to be reasonably fast, however the JVM has to do locking and other checks in order to support multi-threading. It has to do this for each read() call.
>
> Rather than implement the buffering directly in the compare methods, it might be better to implement a generic, non-threadsafe filter that can be used in situations such as these.
>
> The original code should then work without any change other than to add the filter.
Hi.
I used the filter idea you mentioned and re-write another implementation, named contentEqualsIgnoreEOLNew2.
You can see it in the latest commit.
A fast performance test shows that it is SLOWER than my giant function (called contentEqualsIgnoreEOLNew1), but still very much FASTER than the original implementation in commons-io.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -1147,31 +1201,43 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
}
continue;
default:
- if (lastState1 == LastState.r) {
- lastState1 = LastState.newLine;
+ // if input2's next is normal.
+ // if input1's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState1 == LastState.R) {
+ lastState1 = LastState.NEW_LINE;
nowCheck1++;
continue;
} else {
return false;
}
}
default:
+ // if input1's next is normal.
switch (charArray2[nowCheck2]) {
case '\n':
- if (lastState2 == LastState.r) {
- lastState2 = LastState.newLine;
+ // if input2's next is '\n'.
+ // if input2's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState2 == LastState.R) {
+ lastState2 = LastState.NEW_LINE;
nowCheck2++;
continue;
} else {
return false;
}
case '\r':
+ // if input2's next is '\r'.
+ // illegal.
return false;
default:
+ // if input2's next is normal.
+ // if equal then legal.
+ // otherwise illegal.
if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
return false;
}
- lastState1 = lastState2 = LastState.normal;
+ lastState1 = lastState2 = LastState.NORMAL;
nowCheck1++;
nowCheck2++;
continue;
Review comment:
> It occurs to me that the code is effectively buffering the output from a BufferedReader (or BufferedInputStream).
> One would expect these classes to be reasonably fast, however the JVM has to do locking and other checks in order to support multi-threading. It has to do this for each read() call.
>
> Rather than implement the buffering directly in the compare methods, it might be better to implement a generic, non-threadsafe filter that can be used in situations such as these.
>
> The original code should then work without any change other than to add the filter.
Hi.
I used the filter idea you mentioned and re-write another implementation, named contentEqualsIgnoreEOLNew2.
You can see it in the latest commit.
A fast performance test shows that it is SLOWER than my giant function (called contentEqualsIgnoreEOLNew1), but still very much FASTER than the original implementation in commons-io.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -1147,31 +1201,43 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
}
continue;
default:
- if (lastState1 == LastState.r) {
- lastState1 = LastState.newLine;
+ // if input2's next is normal.
+ // if input1's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState1 == LastState.R) {
+ lastState1 = LastState.NEW_LINE;
nowCheck1++;
continue;
} else {
return false;
}
}
default:
+ // if input1's next is normal.
switch (charArray2[nowCheck2]) {
case '\n':
- if (lastState2 == LastState.r) {
- lastState2 = LastState.newLine;
+ // if input2's next is '\n'.
+ // if input2's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState2 == LastState.R) {
+ lastState2 = LastState.NEW_LINE;
nowCheck2++;
continue;
} else {
return false;
}
case '\r':
+ // if input2's next is '\r'.
+ // illegal.
return false;
default:
+ // if input2's next is normal.
+ // if equal then legal.
+ // otherwise illegal.
if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
return false;
}
- lastState1 = lastState2 = LastState.normal;
+ lastState1 = lastState2 = LastState.NORMAL;
nowCheck1++;
nowCheck2++;
continue;
Review comment:
> It occurs to me that the code is effectively buffering the output from a BufferedReader (or BufferedInputStream).
Yes, this is the main trick.
For that two not-so-long function, I used another trick to make it faster with same checkIndex for two buffers.
> One would expect these classes to be reasonably fast, however the JVM has to do locking and other checks in order to support multi-threading. It has to do this for each read() call.
>
> Rather than implement the buffering directly in the compare methods, it might be better to implement a generic, non-threadsafe filter that can be used in situations such as these.
>
> The original code should then work without any change other than to add the filter.
I used the filter idea you mentioned and re-write another implementation, named contentEqualsIgnoreEOLNew2.
You can see it in the latest commit.
A fast performance test shows that it is SLOWER than my giant function (called contentEqualsIgnoreEOLNew1), but still very much FASTER than the original implementation in commons-io.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -1147,31 +1201,43 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
}
continue;
default:
- if (lastState1 == LastState.r) {
- lastState1 = LastState.newLine;
+ // if input2's next is normal.
+ // if input1's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState1 == LastState.R) {
+ lastState1 = LastState.NEW_LINE;
nowCheck1++;
continue;
} else {
return false;
}
}
default:
+ // if input1's next is normal.
switch (charArray2[nowCheck2]) {
case '\n':
- if (lastState2 == LastState.r) {
- lastState2 = LastState.newLine;
+ // if input2's next is '\n'.
+ // if input2's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState2 == LastState.R) {
+ lastState2 = LastState.NEW_LINE;
nowCheck2++;
continue;
} else {
return false;
}
case '\r':
+ // if input2's next is '\r'.
+ // illegal.
return false;
default:
+ // if input2's next is normal.
+ // if equal then legal.
+ // otherwise illegal.
if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
return false;
}
- lastState1 = lastState2 = LastState.normal;
+ lastState1 = lastState2 = LastState.NORMAL;
nowCheck1++;
nowCheck2++;
continue;
Review comment:
> It occurs to me that the code is effectively buffering the output from a BufferedReader (or BufferedInputStream).
Yes, this is the main trick.
For that two not-so-long function, I used another trick to make it faster with same checkIndex for two buffers.
> One would expect these classes to be reasonably fast, however the JVM has to do locking and other checks in order to support multi-threading. It has to do this for each read() call.
>
> Rather than implement the buffering directly in the compare methods, it might be better to implement a generic, non-threadsafe filter that can be used in situations such as these.
>
> The original code should then work without any change other than to add the filter.
I used the filter idea you mentioned and re-write another implementation, named contentEqualsIgnoreEOLNew2.
You can see it in the latest commit.
A fast performance test shows that it is SLOWER than my giant function (called contentEqualsIgnoreEOLNew1), but still very much FASTER than the original implementation in commons-io.
And I worried if we really split it out to class, it may become even slower.
(still, faster than original, of course.)
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -814,9 +814,21 @@ public static boolean contentEquals(final Reader input1, final Reader input2)
}
private enum LastState {
- r,
- normal,
- newLine;
+ /**
+ * If last char is '\r'.
+ */
+ R,
+
+ /**
+ * If last char is not '\r' nor '\n'.
+ */
+ NORMAL,
+
+ /**
+ * If we just moved to a new line.
+ * It might sounds weird but after you see the codes you can know it.
+ */
+ NEW_LINE;
Review comment:
@sebbASF I changed the enum to int, and get even better performance result, so I deleted that enum.
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -882,15 +894,26 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
nowPos2 += nowRead2;
}
}
+
Review comment:
@sebbASF
well it shouldn't in anyway...
If it really happened,I think there must bugs in jvm,and I don't think using >= can make things back to correct...
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -1147,31 +1201,43 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
}
continue;
default:
- if (lastState1 == LastState.r) {
- lastState1 = LastState.newLine;
+ // if input2's next is normal.
+ // if input1's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState1 == LastState.R) {
+ lastState1 = LastState.NEW_LINE;
nowCheck1++;
continue;
} else {
return false;
}
}
default:
+ // if input1's next is normal.
switch (charArray2[nowCheck2]) {
case '\n':
- if (lastState2 == LastState.r) {
- lastState2 = LastState.newLine;
+ // if input2's next is '\n'.
+ // if input2's last is '\r', then it can become "\r\n", then legal.
+ // otherwise illegal.
+ if (lastState2 == LastState.R) {
+ lastState2 = LastState.NEW_LINE;
nowCheck2++;
continue;
} else {
return false;
}
case '\r':
+ // if input2's next is '\r'.
+ // illegal.
return false;
default:
+ // if input2's next is normal.
+ // if equal then legal.
+ // otherwise illegal.
if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
return false;
}
- lastState1 = lastState2 = LastState.normal;
+ lastState1 = lastState2 = LastState.NORMAL;
nowCheck1++;
nowCheck2++;
continue;
Review comment:
@sebbASF
> Most of the comments above are unnecessary if the variables are well named and commented.
> What is missing is how the algorithm works, and what the various combinations of state actually mean in terms of the algorithm.
I added more comments, please have a look.
> I'm not clear why the code sometimes checks for '\r' and sometimes uses the enum.
> Why not use a variable containing the last character?
because in some cases even if last character is '\r', we should treated it as a new line start.
for example, in
"aaaaa\r\naaaaa" and "aaaaa\raaaaa",
although in second String it is \r, it still need to treat it as new line, so new line here not mean LF, but "new line".
though, we can use an int instead of enum.
I tried, and that can work, and even be faster than the enum version....
> I suspect the code could be much simplified by using a suitable filter on the inputs.
Yes I think so. But it will be a performance cost in doing so.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-753328011
> @garydgregory
> got it.
> new added class renamed as:
>
> org.apache.commons.io.input.buffer.UnsyncBufferedInputStream
> org.apache.commons.io.input.buffer.UnsyncBufferedReader
> org.apache.commons.io.input.buffer.LineEndUnifiedBufferedReader
The Java folks make it sounds like these are superfluous in https://bugs.openjdk.java.net/browse/JDK-4097272, so I think we need to see a performance test that shows there is a clear performance benefit to adding those as valuable on their own.
Perhaps our addition of `UnsynchronizedByteArrayInputStream` was a mistake.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462220545
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -790,16 +842,342 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader br1 = toBufferedReader(input1);
- final BufferedReader br2 = toBufferedReader(input2);
- String line1 = br1.readLine();
- String line2 = br2.readLine();
- while (line1 != null && line1.equals(line2)) {
- line1 = br1.readLine();
- line2 = br2.readLine();
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1 = 0;
+ int nowPos2 = 0;
+ int nowRead1;
+ int nowRead2;
+ int nowCheck1 = 0;
+ int nowCheck2 = 0;
+ boolean readEnd1 = false;
+ boolean readEnd2 = false;
+ LastState lastState1 = LastState.newLine;
+ LastState lastState2 = LastState.newLine;
+ while (true) {
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd1) {
+ if (readEnd2) {
+ return true;
+ } else {
+ switch (lastState1) {
+ case r:
+ case newLine:
+ switch (lastState2) {
+ case r:
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState2) {
+ case normal:
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd2) {
+ return true;
+ } else if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ case '\n':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+ } else if (readEnd2) {
+ switch (lastState2) {
+ case r:
+ case newLine:
+ switch (lastState1) {
+ case r:
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState1) {
+ case normal:
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (readEnd1) {
+ return true;
+ } else if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ lastState1 = lastState2 = LastState.r;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck2++;
+ if (readEnd1) {
+ continue;
+ }
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ }
+ continue;
+ default:
+ return false;
+ }
+ case '\n':
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ if (readEnd2) {
+ continue;
+ }
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ }
+ continue;
+ default:
+ if (lastState1 == LastState.r) {
+ lastState1 = LastState.newLine;
+ nowCheck1++;
+ continue;
+ } else {
+ return false;
+ }
+ }
+ default:
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ if (lastState2 == LastState.r) {
+ lastState2 = LastState.newLine;
+ nowCheck2++;
+ continue;
+ } else {
+ return false;
+ }
+ case '\r':
+ return false;
+ default:
+ if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
+ return false;
+ }
+ lastState1 = lastState2 = LastState.normal;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ }
+ }
}
Review comment:
This is one giant complex method, wow. I can't see wanting to maintain it myself. I am not sure if exploding methods like this from simple, short, and easy to understand to complex, large, and hard to grok is what we want. To make this worse from a maintenance POV, there are zero comments in hundreds of lines of code; I can imagine someone seeing this as an opportunity to refactoring all the the repeated patterns into reusable methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r465616775
##########
File path: src/test/java/org/apache/commons/io/IOUtilsTestCase.java
##########
@@ -311,28 +314,38 @@ public synchronized void close() throws IOException {
assertTrue(IOUtils.contentEqualsIgnoreEOL(input1, input1));
}
- Reader r1;
- Reader r2;
-
- r1 = new CharArrayReader("".toCharArray());
- r2 = new CharArrayReader("".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("1".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("1".toCharArray());
- r2 = new CharArrayReader("2".toCharArray());
- assertFalse(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("123\rabc".toCharArray());
- r2 = new CharArrayReader("123\nabc".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
-
- r1 = new CharArrayReader("321".toCharArray());
- r2 = new CharArrayReader("321\r\n".toCharArray());
- assertTrue(IOUtils.contentEqualsIgnoreEOL(r1, r2));
+ testSingleEOL("", "", true);
+ testSingleEOL("", "\n", true);
+ testSingleEOL("", "\r", true);
+ testSingleEOL("", "\r\n", true);
+ testSingleEOL("", "\r\r", false);
+ testSingleEOL("", "\n\n", false);
+ testSingleEOL("1", "1", true);
+ testSingleEOL("1", "2", false);
+ testSingleEOL("123\rabc", "123\nabc", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\naabb", false);
+ testSingleEOL("321", "321\n", true);
+ testSingleEOL("321", "321\r", true);
+ testSingleEOL("321", "321\r\n", true);
+ testSingleEOL("321", "321\r\r", false);
+ testSingleEOL("321", "321\n\r", false);
+ testSingleEOL("321\n", "321", true);
+ testSingleEOL("321\n", "321\n\r", false);
+ testSingleEOL("321\n", "321\r\n", true);
+ testSingleEOL("321\r", "321\r\n", true);
+ testSingleEOL("321\r\n", "321\r\n\r", false);
+ testSingleEOL("123", "1234", false);
+ }
+
Review comment:
Issues with existing tests are should be handled by a separate PR.
For new code, I would only expect to see new tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] garydgregory commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
garydgregory commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-640983198
Just FYI, your expectations for 72 hour consensus might be slightly off, it usually applies to certain kinds of votes, not parts of PRs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669144943
> Also, the existing prefix we use is Undynchronized, not Unsync. Gary
> Undynchronized should presumably be Unsynchronized?
>
> However, I think that makes the class names rather long.
> I think Unsync is clear enough as a prefix (and avoids the issue of whether to use -ized or -ised)
I'm neutral on the naming question.
If you get an agreement about it, I'm glad to rename the two classes according to the result.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] sebbASF commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
sebbASF commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-669139532
Undynchronized should presumably be Unsynchronized?
However, I think that makes the class names rather long.
I think Unsync is clear enough as a prefix (and avoids the issue of whether to use -ized or -ised)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32474365)
Coverage increased (+0.5%) to 90.204% when pulling **0927d5b859e222109290e57abb272d1a5e0b0c38 on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-637035997
@garydgregory
Performance test done.
**Conclusion:**
In cases for small content readers (a File), the new function is slightly faster.
but in cases for large content (a String), the new function can be 6.5 times faster.
Come to re-think of it, it MIGHT because, the FileReader is slow, so the delay of that function is kindof, hided,
But StringReader is faster, so the difference is more significant.
```
D:\workspace\commons-io>mvn -Pbenchmark
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< commons-io:commons-io >------------------------
[INFO] Building Apache Commons IO 2.7.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ commons-io ---
[INFO] Deleting D:\workspace\commons-io\target
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[copy] Copying 2 files to D:\workspace\commons-io\target\apidocs\META-INF
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ commons-io ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ commons-io ---
[INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] Working directory: D:\workspace\commons-io
[INFO] Storing buildNumber: 73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 at timestamp: 1591034434847
[INFO] Storing buildScmBranch: refine_contentEquals
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\workspace\commons-io\src\main\resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\classes
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Recompile with -Xlint:deprecation for details.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOException
List.java uses unchecked or unsafe operations.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-bundle-plugin:4.2.1:manifest (bundle-manifest) @ commons-io ---
[INFO]
[INFO] --- animal-sniffer-maven-plugin:1.18:check (checkAPIcompatibility) @ commons-io ---
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java18:1.0
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:158: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:158: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\TeeReader.java:162: Covariant return type change detected: java.nio.Buffer java.nio.Char
Buffer.limit(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:125: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.flip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:127: Covariant return type change detected: java.nio.Buffer java.
nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:207: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.position(int) has been changed to java.nio.CharBuffer java.nio.CharBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:209: Covariant return type change detected: java.nio.Buffer java.
nio.CharBuffer.flip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\ReaderInputStream.java:213: Covariant return type change detected: java.nio.Buffer java.
nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:74: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:128: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:222: Covariant return type change detected: java.nio.Buffer
java.nio.CharBuffer.mark() has been changed to java.nio.CharBuffer java.nio.CharBuffer.mark()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:223: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.mark() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.mark()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:245: Covariant return type change detected: java.nio.Buffer
java.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:246: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.rewind() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:247: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:249: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.rewind() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:250: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\input\CharSequenceInputStream.java:258: Covariant return type change detected: java.nio.Buffer
java.nio.ByteBuffer.position(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:283: Covariant return type change detected: java.nio.Buffer jav
a.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:309: Covariant return type change detected: java.nio.Buffer jav
a.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:331: Covariant return type change detected: java.nio.Buffer jav
a.nio.ByteBuffer.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\output\WriterOutputStream.java:340: Covariant return type change detected: java.nio.Buffer jav
a.nio.CharBuffer.rewind() has been changed to java.nio.CharBuffer java.nio.CharBuffer.rewind()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:987: Covariant return type change detected: java.nio.Buffer java.nio.CharBuffer.f
lip() has been changed to java.nio.CharBuffer java.nio.CharBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:2008: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffer.
position(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.position(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOUtils.java:2009: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffer.
limit(int) has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.limit(int)
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1094: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.flip() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.flip()
[INFO] D:\workspace\commons-io\src\main\java\org\apache\commons\io\FileUtils.java:1096: Covariant return type change detected: java.nio.Buffer java.nio.ByteBuffe
r.clear() has been changed to java.nio.ByteBuffer java.nio.ByteBuffer.clear()
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] Copying 45 resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\test-classes
[INFO] /D:/workspace/commons-io/src/test/java/org/apache/commons/io/output/FileWriterWithEncodingTest.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/test/java/org/apache/commons/io/output/FileWriterWithEncodingTest.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (prepare-agent) @ commons-io ---
[INFO] argLine set to -javaagent:C:\\Users\\xenoa\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.5\\org.jacoco.agent-0.8.5-runtime.jar=destfile=D:\\workspa
ce\\commons-io\\target\\jacoco.exec
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ commons-io ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:exec (benchmark) @ commons-io ---
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/C:/Users/xenoa/.m2/repository/org/openjdk/jmh/jmh-core/1.21/jmh-core-1.21.jar) to field j
ava.io.PrintStream.charOut
WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew
# Run progress: 0.00% complete, ETA 00:33:20
# Fork: 1 of 5
# Warmup Iteration 1: 8403699.664 ns/op
# Warmup Iteration 2: 8342097.081 ns/op
# Warmup Iteration 3: 7431033.061 ns/op
# Warmup Iteration 4: 7714089.746 ns/op
# Warmup Iteration 5: 7338344.241 ns/op
Iteration 1: 7459707.308 ns/op
Iteration 2: 8792593.849 ns/op
Iteration 3: 10726891.104 ns/op
Iteration 4: 8550254.786 ns/op
Iteration 5: 8001410.312 ns/op
# Run progress: 5.00% complete, ETA 00:31:55
# Fork: 2 of 5
# Warmup Iteration 1: 7848885.804 ns/op
# Warmup Iteration 2: 7489775.075 ns/op
# Warmup Iteration 3: 7328874.652 ns/op
# Warmup Iteration 4: 7308097.663 ns/op
# Warmup Iteration 5: 7298969.001 ns/op
Iteration 1: 7347818.649 ns/op
Iteration 2: 7251497.464 ns/op
Iteration 3: 7366967.231 ns/op
Iteration 4: 7420473.516 ns/op
Iteration 5: 7517702.630 ns/op
# Run progress: 10.00% complete, ETA 00:30:14
# Fork: 3 of 5
# Warmup Iteration 1: 7403000.592 ns/op
# Warmup Iteration 2: 7237561.577 ns/op
# Warmup Iteration 3: 7328035.212 ns/op
# Warmup Iteration 4: 7310369.686 ns/op
# Warmup Iteration 5: 7258066.255 ns/op
Iteration 1: 7387172.526 ns/op
Iteration 2: 7278674.400 ns/op
Iteration 3: 7307894.887 ns/op
Iteration 4: 7363340.103 ns/op
Iteration 5: 7241780.811 ns/op
# Run progress: 15.00% complete, ETA 00:28:33
# Fork: 4 of 5
# Warmup Iteration 1: 7549794.264 ns/op
# Warmup Iteration 2: 7209186.671 ns/op
# Warmup Iteration 3: 7680527.859 ns/op
# Warmup Iteration 4: 7870276.336 ns/op
# Warmup Iteration 5: 8008019.376 ns/op
Iteration 1: 7868122.956 ns/op
Iteration 2: 8014526.603 ns/op
Iteration 3: 7610742.998 ns/op
Iteration 4: 7339908.804 ns/op
Iteration 5: 7466605.970 ns/op
# Run progress: 20.00% complete, ETA 00:26:52
# Fork: 5 of 5
# Warmup Iteration 1: 8062262.933 ns/op
# Warmup Iteration 2: 7888885.647 ns/op
# Warmup Iteration 3: 8092374.353 ns/op
# Warmup Iteration 4: 7535815.813 ns/op
# Warmup Iteration 5: 8376873.032 ns/op
Iteration 1: 7641931.603 ns/op
Iteration 2: 8134969.268 ns/op
Iteration 3: 8823474.603 ns/op
Iteration 4: 9548556.489 ns/op
Iteration 5: 8888150.622 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew":
7934046.780 ?99.9%) 644395.868 ns/op [Average]
(min, avg, max) = (7241780.811, 7934046.780, 10726891.104), stdev = 860250.047
CI (99.9%): [7289650.912, 8578442.648] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2
# Run progress: 25.00% complete, ETA 00:25:12
# Fork: 1 of 5
# Warmup Iteration 1: 427439412.500 ns/op
# Warmup Iteration 2: 411941108.000 ns/op
# Warmup Iteration 3: 415405856.000 ns/op
# Warmup Iteration 4: 408246064.000 ns/op
# Warmup Iteration 5: 381986007.407 ns/op
Iteration 1: 372559803.704 ns/op
Iteration 2: 372221244.444 ns/op
Iteration 3: 394561276.923 ns/op
Iteration 4: 410318660.000 ns/op
Iteration 5: 417270562.500 ns/op
# Run progress: 30.00% complete, ETA 00:23:36
# Fork: 2 of 5
# Warmup Iteration 1: 385519323.077 ns/op
# Warmup Iteration 2: 341417310.000 ns/op
# Warmup Iteration 3: 337620636.667 ns/op
# Warmup Iteration 4: 337741280.000 ns/op
# Warmup Iteration 5: 322947274.194 ns/op
Iteration 1: 334579670.000 ns/op
Iteration 2: 324313664.516 ns/op
Iteration 3: 349864203.448 ns/op
Iteration 4: 345000172.414 ns/op
Iteration 5: 338797430.000 ns/op
# Run progress: 35.00% complete, ETA 00:21:56
# Fork: 3 of 5
# Warmup Iteration 1: 424080325.000 ns/op
# Warmup Iteration 2: 409074288.000 ns/op
# Warmup Iteration 3: 397216476.923 ns/op
# Warmup Iteration 4: 379384881.481 ns/op
# Warmup Iteration 5: 406599224.000 ns/op
Iteration 1: 344006786.667 ns/op
Iteration 2: 318393962.500 ns/op
Iteration 3: 326383164.516 ns/op
Iteration 4: 326728512.903 ns/op
Iteration 5: 339791536.667 ns/op
# Run progress: 40.00% complete, ETA 00:20:17
# Fork: 4 of 5
# Warmup Iteration 1: 359791750.000 ns/op
# Warmup Iteration 2: 344011470.000 ns/op
# Warmup Iteration 3: 317443868.750 ns/op
# Warmup Iteration 4: 366668921.429 ns/op
# Warmup Iteration 5: 340980226.667 ns/op
Iteration 1: 362224464.286 ns/op
Iteration 2: 329631980.645 ns/op
Iteration 3: 328959990.323 ns/op
Iteration 4: 331093680.645 ns/op
Iteration 5: 328042438.710 ns/op
# Run progress: 45.00% complete, ETA 00:18:37
# Fork: 5 of 5
# Warmup Iteration 1: 328533070.968 ns/op
# Warmup Iteration 2: 321877478.125 ns/op
# Warmup Iteration 3: 315723571.875 ns/op
# Warmup Iteration 4: 323838003.226 ns/op
# Warmup Iteration 5: 325582848.387 ns/op
Iteration 1: 322872964.516 ns/op
Iteration 2: 320972165.625 ns/op
Iteration 3: 322146753.125 ns/op
Iteration 4: 320983556.250 ns/op
Iteration 5: 332471216.129 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2":
344567594.458 ?99.9%) 21104733.591 ns/op [Average]
(min, avg, max) = (318393962.500, 344567594.458, 417270562.500), stdev = 28174215.532
CI (99.9%): [323462860.868, 365672328.049] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld
# Run progress: 50.00% complete, ETA 00:16:57
# Fork: 1 of 5
# Warmup Iteration 1: 8040772.910 ns/op
# Warmup Iteration 2: 7850546.468 ns/op
# Warmup Iteration 3: 8366507.191 ns/op
# Warmup Iteration 4: 8919272.193 ns/op
# Warmup Iteration 5: 8934492.411 ns/op
Iteration 1: 9849140.059 ns/op
Iteration 2: 10280016.496 ns/op
Iteration 3: 9377572.540 ns/op
Iteration 4: 8634857.550 ns/op
Iteration 5: 8796502.111 ns/op
# Run progress: 55.00% complete, ETA 00:15:14
# Fork: 2 of 5
# Warmup Iteration 1: 9196890.441 ns/op
# Warmup Iteration 2: 8651530.769 ns/op
# Warmup Iteration 3: 9130613.504 ns/op
# Warmup Iteration 4: 9528571.075 ns/op
# Warmup Iteration 5: 8603469.046 ns/op
Iteration 1: 8649056.612 ns/op
Iteration 2: 8740562.707 ns/op
Iteration 3: 8636735.807 ns/op
Iteration 4: 8658414.545 ns/op
Iteration 5: 9077745.554 ns/op
# Run progress: 60.00% complete, ETA 00:13:32
# Fork: 3 of 5
# Warmup Iteration 1: 9210640.074 ns/op
# Warmup Iteration 2: 7950668.124 ns/op
# Warmup Iteration 3: 8049352.132 ns/op
# Warmup Iteration 4: 7906520.949 ns/op
# Warmup Iteration 5: 8193835.545 ns/op
Iteration 1: 7833574.706 ns/op
Iteration 2: 8025837.640 ns/op
Iteration 3: 8621776.034 ns/op
Iteration 4: 8516147.915 ns/op
Iteration 5: 8379908.710 ns/op
# Run progress: 65.00% complete, ETA 00:11:50
# Fork: 4 of 5
# Warmup Iteration 1: 8810415.141 ns/op
# Warmup Iteration 2: 8794569.859 ns/op
# Warmup Iteration 3: 8730751.745 ns/op
# Warmup Iteration 4: 8563348.202 ns/op
# Warmup Iteration 5: 8946487.757 ns/op
Iteration 1: 8798953.725 ns/op
Iteration 2: 9077297.461 ns/op
Iteration 3: 9203544.250 ns/op
Iteration 4: 10801144.492 ns/op
Iteration 5: 10137045.694 ns/op
# Run progress: 70.00% complete, ETA 00:10:09
# Fork: 5 of 5
# Warmup Iteration 1: 10034710.933 ns/op
# Warmup Iteration 2: 7852066.954 ns/op
# Warmup Iteration 3: 8036246.265 ns/op
# Warmup Iteration 4: 8208804.184 ns/op
# Warmup Iteration 5: 8919215.241 ns/op
Iteration 1: 7703625.019 ns/op
Iteration 2: 8173784.069 ns/op
Iteration 3: 8476930.000 ns/op
Iteration 4: 9254641.073 ns/op
Iteration 5: 8732920.419 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld":
8897509.408 ?99.9%) 558227.023 ns/op [Average]
(min, avg, max) = (7703625.019, 8897509.408, 10801144.492), stdev = 745217.105
CI (99.9%): [8339282.385, 9455736.431] (assumes normal distribution)
# JMH version: 1.21
# VM version: JDK 13.0.2, OpenJDK 64-Bit Server VM, 13.0.2+8
# VM invoker: C:\jdk-13.0.2+8\bin\java.exe
# VM options: <none>
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2
# Run progress: 75.00% complete, ETA 00:08:27
# Fork: 1 of 5
# Warmup Iteration 1: 2274597420.000 ns/op
# Warmup Iteration 2: 2161414140.000 ns/op
# Warmup Iteration 3: 2060648900.000 ns/op
# Warmup Iteration 4: 2034553880.000 ns/op
# Warmup Iteration 5: 2036716060.000 ns/op
Iteration 1: 2039811500.000 ns/op
Iteration 2: 2029332780.000 ns/op
Iteration 3: 2048828760.000 ns/op
Iteration 4: 2167898680.000 ns/op
Iteration 5: 2375598240.000 ns/op
# Run progress: 80.00% complete, ETA 00:06:47
# Fork: 2 of 5
# Warmup Iteration 1: 2302730460.000 ns/op
# Warmup Iteration 2: 2169550200.000 ns/op
# Warmup Iteration 3: 2183069220.000 ns/op
# Warmup Iteration 4: 2219912760.000 ns/op
# Warmup Iteration 5: 2276130160.000 ns/op
Iteration 1: 2256782140.000 ns/op
Iteration 2: 2212519940.000 ns/op
Iteration 3: 2194957540.000 ns/op
Iteration 4: 2139474860.000 ns/op
Iteration 5: 2239168680.000 ns/op
# Run progress: 85.00% complete, ETA 00:05:07
# Fork: 3 of 5
# Warmup Iteration 1: 2871005625.000 ns/op
# Warmup Iteration 2: 2234621780.000 ns/op
# Warmup Iteration 3: 2057113240.000 ns/op
# Warmup Iteration 4: 2106691240.000 ns/op
# Warmup Iteration 5: 2239427440.000 ns/op
Iteration 1: 2221194720.000 ns/op
Iteration 2: 2252754040.000 ns/op
Iteration 3: 2362948520.000 ns/op
Iteration 4: 2233297260.000 ns/op
Iteration 5: 2289872180.000 ns/op
# Run progress: 90.00% complete, ETA 00:03:25
# Fork: 4 of 5
# Warmup Iteration 1: 2316297000.000 ns/op
# Warmup Iteration 2: 2186447840.000 ns/op
# Warmup Iteration 3: 2186496100.000 ns/op
# Warmup Iteration 4: 2210486500.000 ns/op
# Warmup Iteration 5: 2248586640.000 ns/op
Iteration 1: 2228412480.000 ns/op
Iteration 2: 2293236620.000 ns/op
Iteration 3: 2300933160.000 ns/op
Iteration 4: 2050526440.000 ns/op
Iteration 5: 2103876620.000 ns/op
# Run progress: 95.00% complete, ETA 00:01:43
# Fork: 5 of 5
# Warmup Iteration 1: 2231988940.000 ns/op
# Warmup Iteration 2: 2132075180.000 ns/op
# Warmup Iteration 3: 2107424340.000 ns/op
# Warmup Iteration 4: 2244625860.000 ns/op
# Warmup Iteration 5: 2212587240.000 ns/op
Iteration 1: 2358689240.000 ns/op
Iteration 2: 2360920240.000 ns/op
Iteration 3: 2304193360.000 ns/op
Iteration 4: 2120541520.000 ns/op
Iteration 5: 2225049120.000 ns/op
Result "org.apache.commons.io.performance.IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2":
2216432745.600 ?99.9%) 79521556.878 ns/op [Average]
(min, avg, max) = (2029332780.000, 2216432745.600, 2375598240.000), stdev = 106159003.301
CI (99.9%): [2136911188.722, 2295954302.478] (assumes normal distribution)
# Run complete. Total time: 00:34:37
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
IOUtilsContentEqualsPerformanceTest.testContentEqualsForFileNew avgt 25 7934046.780 ? 644395.868 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsNew2 avgt 25 344567594.458 ?21104733.591 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsOld avgt 25 8897509.408 ? 558227.023 ns/op
IOUtilsContentEqualsPerformanceTest.testContentEqualsOld2 avgt 25 2216432745.600 ?79521556.878 ns/op
Benchmark result is saved to target/jmh-result.org.apache.json
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.8.2:attach-descriptor (attach-descriptor) @ commons-io ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:test-jar (default) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.0:jar-no-fork (create-source-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-sources.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.0:test-jar-no-fork (create-source-jar) @ commons-io ---
[INFO] Building jar: D:\workspace\commons-io\target\commons-io-2.7.1-SNAPSHOT-test-sources.jar
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:check (check) @ commons-io ---
[INFO] Skipping JaCoCo execution due to missing execution data file:D:\workspace\commons-io\target\jacoco.exec
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (default-cli) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] >>> clirr-maven-plugin:2.8:check (default-cli) > compile @ commons-io >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ commons-io ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ commons-io ---
[INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] Working directory: D:\workspace\commons-io
[INFO] Storing buildNumber: 73dc0ff3e6d79ab88f1ed0d253d52cb6da0dffa5 at timestamp: 1591036535724
[INFO] Storing buildScmBranch: refine_contentEquals
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ commons-io ---
[INFO] Using 'iso-8859-1' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory D:\workspace\commons-io\src\main\resources
[INFO] Copying 2 resources to META-INF
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ commons-io ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 158 source files to D:\workspace\commons-io\target\classes
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Some input files use or override a deprecated API.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/input/ClassLoaderObjectInputStream.java: Recompile with -Xlint:deprecation for details.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: D:\workspace\commons-io\src\main\java\org\apache\commons\io\IOException
List.java uses unchecked or unsafe operations.
[INFO] /D:/workspace/commons-io/src/main/java/org/apache/commons/io/IOExceptionList.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] <<< clirr-maven-plugin:2.8:check (default-cli) < compile @ commons-io <<<
[INFO]
[INFO]
[INFO] --- clirr-maven-plugin:2.8:check (default-cli) @ commons-io ---
[INFO] Comparing to version: 2.7
[INFO] Succeeded with 0 errors; 0 warnings; and 0 other changes.
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.0:check (default-cli) @ commons-io ---
[INFO]
[INFO] >>> maven-javadoc-plugin:3.1.1:javadoc (default-cli) > generate-sources @ commons-io >>>
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ commons-io ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-3) @ commons-io ---
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ commons-io ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Parsing exclusions from D:\workspace\commons-io\.gitignore
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 73 implicit excludes (use -debug for more details).
[INFO] 12 explicit excludes (use -debug for more details).
[INFO] 373 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 364 licenses.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ commons-io ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @ commons-io ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] <<< maven-javadoc-plugin:3.1.1:javadoc (default-cli) < generate-sources @ commons-io <<<
[INFO]
[INFO]
[INFO] --- maven-javadoc-plugin:3.1.1:javadoc (default-cli) @ commons-io ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 35:31 min
[INFO] Finished at: 2020-06-02T02:35:59+08:00
[INFO] ------------------------------------------------------------------------
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-640937650
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32560038)
Coverage increased (+0.4%) to 90.17% when pulling **a247cfc7e50d6afa5d01597051e258ca4762ec9f on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636994263
@garydgregory
> @XenoAmess
>
> * When you create a PR, please do not leave the description empty. It makes it harder to review.
OK
> * If a PR claims a performance improvement, it must back it up; some components do this using JMH microbenchmarks, for example please see Apache Commons BCEL, Crypto, CSV, Lang, Math, RNG.
I have only experiences using Jprofiler and I don't know how to embed a test with performance in junit.
I'll learn about JMH microbenchmarks.
thanks.
update: I learned how to use it, thanks for your recommendation.
also, need I create more tests for testing wether this function is correct?
or tests that already exist, which for the older function, is good enough and I need not create more?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on a change in pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on a change in pull request #118:
URL: https://github.com/apache/commons-io/pull/118#discussion_r462755702
##########
File path: src/main/java/org/apache/commons/io/IOUtils.java
##########
@@ -790,16 +842,342 @@ public static boolean contentEqualsIgnoreEOL(final Reader input1, final Reader i
if (input1 == null ^ input2 == null) {
return false;
}
- final BufferedReader br1 = toBufferedReader(input1);
- final BufferedReader br2 = toBufferedReader(input2);
- String line1 = br1.readLine();
- String line2 = br2.readLine();
- while (line1 != null && line1.equals(line2)) {
- line1 = br1.readLine();
- line2 = br2.readLine();
+ char[] charArray1 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ char[] charArray2 = new char[CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE];
+ int nowPos1 = 0;
+ int nowPos2 = 0;
+ int nowRead1;
+ int nowRead2;
+ int nowCheck1 = 0;
+ int nowCheck2 = 0;
+ boolean readEnd1 = false;
+ boolean readEnd2 = false;
+ LastState lastState1 = LastState.newLine;
+ LastState lastState2 = LastState.newLine;
+ while (true) {
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2, CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd1) {
+ if (readEnd2) {
+ return true;
+ } else {
+ switch (lastState1) {
+ case r:
+ case newLine:
+ switch (lastState2) {
+ case r:
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState2) {
+ case normal:
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ if (readEnd2) {
+ return true;
+ } else if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ }
+ return false;
+ case '\n':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ return readEnd2;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+ } else if (readEnd2) {
+ switch (lastState2) {
+ case r:
+ case newLine:
+ switch (lastState1) {
+ case r:
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ default:
+ return false;
+ }
+ case normal:
+ switch (lastState1) {
+ case normal:
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ if (readEnd1) {
+ return true;
+ } else if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ }
+ return false;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ return readEnd1;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+ default:
+ //shall never enter
+ }
+ }
+
+ switch (charArray1[nowCheck1]) {
+ case '\r':
+ switch (charArray2[nowCheck2]) {
+ case '\r':
+ lastState1 = lastState2 = LastState.r;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\n':
+ nowCheck1++;
+ if (nowPos1 == nowCheck1) {
+ if (nowCheck1 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos1 = nowCheck1 = 0;
+ }
+ do {
+ nowRead1 = input1.read(charArray1, nowPos1,
+CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos1);
+ } while (nowRead1 == 0);
+ if (nowRead1 == -1) {
+ readEnd1 = true;
+ } else {
+ nowPos1 += nowRead1;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck2++;
+ if (readEnd1) {
+ continue;
+ }
+ if (charArray1[nowCheck1] == '\n') {
+ nowCheck1++;
+ }
+ continue;
+ default:
+ return false;
+ }
+ case '\n':
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ case '\r':
+ nowCheck2++;
+ if (nowPos2 == nowCheck2) {
+ if (nowCheck2 == CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE) {
+ nowPos2 = nowCheck2 = 0;
+ }
+ do {
+ nowRead2 = input2.read(charArray2, nowPos2,
+ CONTENT_EQUALS_CHAR_ARRAY_BUFFER_SIZE - nowPos2);
+ } while (nowRead2 == 0);
+ if (nowRead2 == -1) {
+ readEnd2 = true;
+ } else {
+ nowPos2 += nowRead2;
+ }
+ }
+ lastState1 = lastState2 = LastState.newLine;
+ nowCheck1++;
+ if (readEnd2) {
+ continue;
+ }
+ if (charArray2[nowCheck2] == '\n') {
+ nowCheck2++;
+ }
+ continue;
+ default:
+ if (lastState1 == LastState.r) {
+ lastState1 = LastState.newLine;
+ nowCheck1++;
+ continue;
+ } else {
+ return false;
+ }
+ }
+ default:
+ switch (charArray2[nowCheck2]) {
+ case '\n':
+ if (lastState2 == LastState.r) {
+ lastState2 = LastState.newLine;
+ nowCheck2++;
+ continue;
+ } else {
+ return false;
+ }
+ case '\r':
+ return false;
+ default:
+ if (charArray1[nowCheck1] != charArray2[nowCheck2]) {
+ return false;
+ }
+ lastState1 = lastState2 = LastState.normal;
+ nowCheck1++;
+ nowCheck2++;
+ continue;
+ }
+ }
}
Review comment:
@garydgregory @melloware
split done.
now it is shorter...
far shorter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] coveralls edited a comment on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-636980413
[](https://coveralls.io/builds/32407957)
Coverage decreased (-0.04%) to 89.714% when pulling **86a51bb29f9134a1e230f99b5d43cf4da0b477fd on XenoAmess:refine_contentEquals** into **0dbe95715197c3c9b8c983c0b9acbe87d5cee34d on apache:master**.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-748430204
@garydgregory Weekend now. Any news？:)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-637013520
@garydgregory BTW, how can I show the performance result generated?
simply paste it here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-753646208
> > @garydgregory
> > got it.
> > new added class renamed as:
> > org.apache.commons.io.input.buffer.UnsyncBufferedInputStream
> > org.apache.commons.io.input.buffer.UnsyncBufferedReader
> > org.apache.commons.io.input.buffer.LineEndUnifiedBufferedReader
>
> The Java folks make it sounds like these are superfluous in https://bugs.openjdk.java.net/browse/JDK-4097272, so I think we need to see a performance test that shows there is a clear performance benefit to adding those as valuable on their own.
>
> Perhaps our addition of `UnsynchronizedByteArrayInputStream` was a mistake.
@garydgregory
Hi.
I already listed some performance test results 3 months ago, in this thread. please look back and see them.
isn't the performance test in the list enough?
If so , what other types of performance test do you need?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [commons-io] XenoAmess commented on pull request #118: [IO-670] refine IOUtils.contentEquals(Reader, Reader)
Posted by GitBox <gi...@apache.org>.
XenoAmess commented on pull request #118:
URL: https://github.com/apache/commons-io/pull/118#issuecomment-776112582
> Hi @XenoAmess
> Please see git master and rebase. I updated with your changes for InputStream and Reader but not for the EOL APIs. If you still want those in, please rebase and follow the pattern I established with the *Benchmark classes.
> TY! -)
> PS: I gave you credit in the commit message and changes.xml 👍
rebased. benchmark redone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org