You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/11/26 15:21:00 UTC
[jira] [Work logged] (CODEC-265)
java.lang.NegativeArraySizeException
[ https://issues.apache.org/jira/browse/CODEC-265?focusedWorklogId=349834&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-349834 ]
ASF GitHub Bot logged work on CODEC-265:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Nov/19 15:20
Start Date: 26/Nov/19 15:20
Worklog Time Spent: 10m
Work Description: aherbert commented on pull request #30: [CODEC-265] BaseNCodec to expand buffer using overflow conscious code.
URL: https://github.com/apache/commons-codec/pull/30
Re: [CODEC-265] Add a test to encode a 1 Gigabyte byte[].
Added overflow conscious code for buffer expansion during encoding and decoding. This is based on java.util.ArrayList but modified to use a local copy of JDK 1.8 Integer.compareUnsigned(int, int) to compare the new buffer capacity to the required capacity.
Unlike the java.util.ArrayList if the minCapacity exceeds the head room for the maximum array size, the capacity returned is the required minCapacity and not Integer.MAX_VALUE. This allows the test `testEnsureBufferSizeExpandsToBeyondMaxBufferSize` to pass on my machine using:
```
java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
```
If I return Integer.MAX_VALUE for the new capacity then the test fails as my machine cannot allocate arrays of this size. This test has an assume method that tries to allocate the very large raw byte array. If this is not possible then the test is skipped.
This fix allows reinstating Base64 HugeLineSeparator test as the buffer expansion ensures that the minimum allowed capacity is allocated even if doubling the buffer size is not enough.
Note:
The test for allocating the maximum buffer size requires >2GB of RAM. The test for encoding a 1 GB file requires >3GB of RAM. The tests attempt to detect the amount of RAM that can be allocated and skip if necessary. Travis does not run the test for encoding a 1GB file as it does not have 3GB of memory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 349834)
Remaining Estimate: 0h
Time Spent: 10m
> java.lang.NegativeArraySizeException
> --------------------------------------
>
> Key: CODEC-265
> URL: https://issues.apache.org/jira/browse/CODEC-265
> Project: Commons Codec
> Issue Type: Bug
> Affects Versions: 1.13
> Environment: Linux = Ubuntu 18.04.3 LTS
> JDK = 1.8
>
> Reporter: Ingimar
> Assignee: Alex Herbert
> Priority: Critical
> Attachments: NewClientEncodePost.java, Util.java, pom.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi,
> trying to encode a file that is 1GB of size.
> ( linux :
> {code:java}
> fallocate -l 1GB 1gb.zip{code}
> )
> I want to post that file to a RESTful-service, package in JSON.
> *here is the code*
>
>
> {code:java}
> String filePath = "/tmp/1gb.zip";
> System.out.println("\t Post to : ".concat(URL));
> System.out.println("\t file : ".concat(filePath));
> Path path = Paths.get(filePath);
> byte[] bArray = Files.readAllBytes(path);
> // testing commons codec 1.16 (2019-11-05)
> byte[] encodeBase64 = Base64.encodeBase64(bArray);
> final String contentToBeSaved = new String(encodeBase64);
> HttpClient client = HttpClientBuilder.create().build();
> HttpResponse response = null;
> JSONObject metadata = new JSONObject();
> metadata.put("owner", "Ingo");
> metadata.put("access", "public");
> metadata.put("licenseType", "CC BY");
> metadata.put("fileName", "fileName");
> metadata.put("fileDataBase64", contentToBeSaved);
> String metadataFormatted = StringEscapeUtils.unescapeJavaScript(metadata.toString());
> StringEntity entity = new StringEntity(metadataFormatted, ContentType.APPLICATION_JSON);
> HttpPost post = new HttpPost(URL);
> post.setEntity(entity);
> response = client.execute(post);
> HttpEntity responseEntity = response.getEntity();
> String responseFromMediaserver = EntityUtils.toString(responseEntity, "UTF-8");
> System.out.println("\n****");
> System.out.println("Response is : " + responseFromMediaserver);
> JSONObject json = new JSONObject(responseFromMediaserver);
> String uuid = json.getString("uuid");
> System.out.println("UUID is " + uuid);
> {code}
>
>
> # mvn clean package
> # java -Xms512m -Xmx20480m -jar target/mediaClient.jar
> The crasch is in
>
> {code:java}
> byte[] encodeBase64 = Base64.encodeBase64(bArray);{code}
>
> the stacktrace is :
> {code:java}
>
> Starting NewClientEncodePost
> Post to : http://127.0.0.1:8080/MediaServerResteasy/media
> file : /tmp/1gb.zip
> Exception in thread "main" java.lang.NegativeArraySizeException
> at org.apache.commons.codec.binary.BaseNCodec.resizeBuffer(BaseNCodec.java:253)
> at org.apache.commons.codec.binary.BaseNCodec.ensureBufferSize(BaseNCodec.java:269)
> at org.apache.commons.codec.binary.Base64.encode(Base64.java:380)
> at org.apache.commons.codec.binary.BaseNCodec.encode(BaseNCodec.java:451)
> at org.apache.commons.codec.binary.BaseNCodec.encode(BaseNCodec.java:430)
> at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:679)
> at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:642)
> at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:623)
> at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:556)
> at se.nrm.bio.mediaserver.testing.base64.NewClientEncodePost.posting(NewClientEncodePost.java:55)
> at se.nrm.bio.mediaserver.testing.base64.NewClientEncodePost.main(NewClientEncodePost.java:38)
>
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)