You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Andrzej Bialecki (Jira)" <ji...@apache.org> on 2026/09/08 10:01:00 UTC
[jira] [Commented] (SOLR-18415) CrossDC Common: ExposedByteArrayOutputStream optimizes wrong aspect
[ https://issues.apache.org/jira/browse/SOLR-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112646#comment-18112646 ]
Andrzej Bialecki commented on SOLR-18415:
-----------------------------------------
After further analysis this looks somewhat better than I thought, especially for updates that are less than 8192 byte in size (serialized), because for these the array is resized to the actual size of the payload (as accumulated in the {{FastOutputStream}}'s buffer that the {{JavaBinCodec}} uses). In this case no extra padding bytes are wasted.
Since the producer side ({{MirroringUpdateProcessor / KafkaMirroringSink}}) doesn't accumulate individual requests in batches and in a typical scenario most documents are smaller than 8kB serialized, this optimization will mostly hold.
This is not the case when sending larger documents or when sending MirroredConfigSetRequest-s that have multiple content streams. Here's the breakdown:
{noformat}
actualSize=61 byteArraySize=64 waste=3 (4.7% of transmitted bytes)
actualSize=153 byteArraySize=153 waste=0 (0.0% of transmitted bytes)
actualSize=604 byteArraySize=604 waste=0 (0.0% of transmitted bytes)
actualSize=4104 byteArraySize=4104 waste=0 (0.0% of transmitted bytes)
actualSize=6104 byteArraySize=6104 waste=0 (0.0% of transmitted bytes)
actualSize=8104 byteArraySize=8104 waste=0 (0.0% of transmitted bytes)
actualSize=8304 byteArraySize=16464 waste=8160 (49.6% of transmitted bytes)
actualSize=10104 byteArraySize=20064 waste=9960 (49.6% of transmitted bytes)
actualSize=15104 byteArraySize=30064 waste=14960 (49.8% of transmitted bytes)
actualSize=20105 byteArraySize=40066 waste=19961 (49.8% of transmitted bytes)
actualSize=30105 byteArraySize=32768 waste=2663 (8.1% of transmitted bytes)
actualSize=50105 byteArraySize=65536 waste=15431 (23.5% of transmitted bytes)
actualSize=100105 byteArraySize=131072 waste=30967 (23.6% of transmitted bytes)
{noformat}
The {{actualSize}} represents the size of serialized data, {{byteArraySize}} is the size of the returned byte array that contains the serialized data. Up until the boundary of 8192 bytes the optimization works very well, but above this threshold it produces a lot of waste.
I propose to add a conditional in {{ExposedByteArrayOutputStream.byteArray()}} to return the raw buffer only if it's at most 10% larger than the size of serialized data. This will cover all small-doc cases and avoid an array copy for some less egregious cases for larger docs.
Here's the breakdown with the proposed optimization:
{noformat}
actualSize=61 byteArraySize=64 waste=3 (4.7% of transmitted bytes)
actualSize=153 byteArraySize=153 waste=0 (0.0% of transmitted bytes)
actualSize=604 byteArraySize=604 waste=0 (0.0% of transmitted bytes)
actualSize=4104 byteArraySize=4104 waste=0 (0.0% of transmitted bytes)
actualSize=6104 byteArraySize=6104 waste=0 (0.0% of transmitted bytes)
actualSize=8104 byteArraySize=8104 waste=0 (0.0% of transmitted bytes)
actualSize=8304 byteArraySize=8304 waste=0 (0.0% of transmitted bytes)
actualSize=17386 byteArraySize=17386 waste=0 (0.0% of transmitted bytes)
actualSize=24700 byteArraySize=24700 waste=0 (0.0% of transmitted bytes)
actualSize=28550 byteArraySize=28550 waste=0 (0.0% of transmitted bytes)
actualSize=31026 byteArraySize=32768 waste=1742 (5.3% of transmitted bytes)
actualSize=36877 byteArraySize=36877 waste=0 (0.0% of transmitted bytes)
actualSize=43667 byteArraySize=43667 waste=0 (0.0% of transmitted bytes)
actualSize=47067 byteArraySize=47067 waste=0 (0.0% of transmitted bytes)
actualSize=50696 byteArraySize=50696 waste=0 (0.0% of transmitted bytes)
actualSize=57516 byteArraySize=57516 waste=0 (0.0% of transmitted bytes)
actualSize=61110 byteArraySize=65536 waste=4426 (6.8% of transmitted bytes)
actualSize=69145 byteArraySize=69145 waste=0 (0.0% of transmitted bytes)
actualSize=79692 byteArraySize=79692 waste=0 (0.0% of transmitted bytes)
actualSize=89342 byteArraySize=89342 waste=0 (0.0% of transmitted bytes)
actualSize=91734 byteArraySize=91734 waste=0 (0.0% of transmitted bytes)
actualSize=93769 byteArraySize=93769 waste=0 (0.0% of transmitted bytes)
{noformat}
> CrossDC Common: ExposedByteArrayOutputStream optimizes wrong aspect
> -------------------------------------------------------------------
>
> Key: SOLR-18415
> URL: https://issues.apache.org/jira/browse/SOLR-18415
> Project: Solr
> Issue Type: Bug
> Components: module - crossDC
> Affects Versions: 10.0, 9.10.1
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Fix For: main(11.0), 10.1
>
>
> {{MirroredSolrRequestSerializer}} uses a subclass of {{ByteArrayOutputStream}} in an attempt to optimize data transfer by avoiding array copy - instead it passes the whole buffer directly as the output bytes from {{{}serialize(){}}}.
> This sounds like a plausible optimization. However, the initial size of the buffer (32 bytes) is doubled on each overflow as the codec writes to the stream, which means that on average the actual serialized data occupies around half of the buffer, the rest are zeroes. Deserialization will handle these trailing zeros just fine, it's the cost of transmitting roughly twice as much data over the wire that is questionable - this buffer is then passed as-is to the output stream and eventually ends up as network traffic.
> This "optimization" avoids one array copy cost (which reduces GC) but generates multiple costs on the way - not only the empty data has to be sent over the network but also it needs to be stored in source Kafka, in MirrorMaker and in target Kafka, to be finally retrieved by the Consumer (which has to allocate a buffer sufficiently large to fit also the zeroes), only to discard the zeroes during deserialization.
> I propose to get rid of this optimization and use the plain {{ByteArrayOutputStream}} instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org