You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cxf.apache.org by "Lucas Pouzac (JIRA)" <ji...@apache.org> on 2015/01/13 16:10:37 UTC
[jira] [Comment Edited] (CXF-6189) Improve memory usage of UrlUtils
[ https://issues.apache.org/jira/browse/CXF-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275337#comment-14275337 ]
Lucas Pouzac edited comment on CXF-6189 at 1/13/15 3:10 PM:
------------------------------------------------------------
I did some testing and some modifications. For me, I think we need to better optimize the memory usage to be quite a bit slower.
{code:java}
/**
* V2
*/
private static String urlDecode(String value, String enc, boolean isPath) {
int length = 0;
boolean needEncode = false;
for (int i = 0; i < value.length(); i++) {
if (value.charAt(i) == ESCAPE_CHAR) {
length += 1;
needEncode = true;
} else if (value.charAt(i) == PLUS_CHAR) {
needEncode = true;
}
}
if (needEncode) {
final byte[] valueBytes = StringUtils.toBytes(value, enc);
ByteBuffer in = ByteBuffer.wrap(valueBytes);
ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length)); // MODIFY
while (in.hasRemaining()) {
final int b = in.get();
if (!isPath && b == PLUS_CHAR) {
out.put((byte) ' ');
} else if (b == ESCAPE_CHAR) {
try {
final int u = digit16((byte) in.get());
final int l = digit16((byte) in.get());
out.put((byte) ((u << 4) + l));
} catch (final ArrayIndexOutOfBoundsException e) {
throw new RuntimeException("Invalid URL encoding: ", e);
}
} else {
out.put((byte) b);
}
}
out.flip();
return Charset.forName(enc).decode(out).toString();
} else {
return value;
}
}
{code}
For V3, replace
{code:java}
byte[] valueBytes = value.getBytes(enc);
ByteBuffer in = ByteBuffer.wrap(valueBytes);
ByteBuffer out = ByteBuffer.allocate(in.capacity());
{code}
by
{code:java}
ByteBuffer in = Charset.forName(enc).encode(CharBuffer.wrap(value));
ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length));
{code}
Result for v3
- Faster than cxf 3.0.3
- Slower than v1 and v2
- But Memory usage has decreased.
{code}
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3 thrpt 60 219953,443 ± 4342,013 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.profiled thrpt 60 1344,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.total thrpt 60 927,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.profiled thrpt 60 4917,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.total thrpt 60 3388,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin thrpt 60 430193,040 ± 14953,635 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.profiled thrpt 60 1383,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.total thrpt 60 951,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.profiled thrpt 60 5155,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.total thrpt 60 3501,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2 thrpt 60 425701,711 ± 11373,886 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.profiled thrpt 60 1213,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.total thrpt 60 825,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.profiled thrpt 60 4497,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.total thrpt 60 3088,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3 thrpt 60 319130,222 ± 6422,034 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.profiled thrpt 60 589,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.total thrpt 60 388,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.profiled thrpt 60 2085,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.total thrpt 60 1454,000 ± NaN ms
{code}
I think we should use the v3
was (Author: lpouzac):
I did some testing and some modifications. For me, I think we need to better optimize the memory usage to be quite a bit slower.
{code:java}
/**
* V2
*/
private static String urlDecode(String value, String enc, boolean isPath) {
int length = 0;
boolean needEncode = false;
for (int i = 0; i < value.length(); i++) {
if (value.charAt(i) == ESCAPE_CHAR) {
length += 1;
needEncode = true;
} else if (value.charAt(i) == PLUS_CHAR) {
needEncode = true;
}
}
if (needEncode) {
final byte[] valueBytes = StringUtils.toBytes(value, enc);
ByteBuffer in = ByteBuffer.wrap(valueBytes);
ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length)); // HERE
while (in.hasRemaining()) {
final int b = in.get();
if (!isPath && b == PLUS_CHAR) {
out.put((byte) ' ');
} else if (b == ESCAPE_CHAR) {
try {
final int u = digit16((byte) in.get());
final int l = digit16((byte) in.get());
out.put((byte) ((u << 4) + l));
} catch (final ArrayIndexOutOfBoundsException e) {
throw new RuntimeException("Invalid URL encoding: ", e);
}
} else {
out.put((byte) b);
}
}
out.flip();
return Charset.forName(enc).decode(out).toString();
} else {
return value;
}
}
{code}
For V3, replace
{code:java}
byte[] valueBytes = value.getBytes(enc);
ByteBuffer in = ByteBuffer.wrap(valueBytes);
ByteBuffer out = ByteBuffer.allocate(in.capacity());
{code}
by
{code:java}
ByteBuffer in = Charset.forName(enc).encode(CharBuffer.wrap(value));
ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length));
{code}
Result for v3
- Faster than cxf 3.0.3
- Slower than v1 and v2
- But Memory usage has decreased.
{code}
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3 thrpt 60 219953,443 ± 4342,013 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.profiled thrpt 60 1344,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.total thrpt 60 927,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.profiled thrpt 60 4917,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.total thrpt 60 3388,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin thrpt 60 430193,040 ± 14953,635 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.profiled thrpt 60 1383,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.total thrpt 60 951,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.profiled thrpt 60 5155,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.total thrpt 60 3501,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2 thrpt 60 425701,711 ± 11373,886 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.profiled thrpt 60 1213,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.total thrpt 60 825,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.profiled thrpt 60 4497,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.total thrpt 60 3088,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3 thrpt 60 319130,222 ± 6422,034 ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.profiled thrpt 60 589,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.total thrpt 60 388,000 ± NaN counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.profiled thrpt 60 2085,000 ± NaN ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.total thrpt 60 1454,000 ± NaN ms
{code}
I think we should use the v3
> Improve memory usage of UrlUtils
> --------------------------------
>
> Key: CXF-6189
> URL: https://issues.apache.org/jira/browse/CXF-6189
> Project: CXF
> Issue Type: Bug
> Components: Core
> Affects Versions: 3.0.3
> Reporter: Lucas Pouzac
> Labels: performance
> Attachments: jmh.tar.gz, screenshot-1.png
>
>
> When I run load test, I find that the management of encoding parameters of the urls is consumer memory.
> I do not know if it is possible to optimize this part.
> Throughput of load test : 400 query/s
> ~80% GET query with 6 parameters
> ~20% POST query with 6 parameters GET and 1 payload
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)