You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Andrzej Bialecki (Jira)" <ji...@apache.org> on 2026/09/08 10:01:00 UTC

[jira] [Commented] (SOLR-18415) CrossDC Common: ExposedByteArrayOutputStream optimizes wrong aspect

    [ https://issues.apache.org/jira/browse/SOLR-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112646#comment-18112646 ] 

Andrzej Bialecki commented on SOLR-18415:
-----------------------------------------

After further analysis this looks somewhat better than I thought, especially for updates that are less than 8192 byte in size (serialized), because for these the array is resized to the actual size of the payload (as accumulated in the {{FastOutputStream}}'s buffer that the {{JavaBinCodec}} uses). In this case no extra padding bytes are wasted.

Since the producer side ({{MirroringUpdateProcessor / KafkaMirroringSink}}) doesn't accumulate individual requests in batches and in a typical scenario most documents are smaller than 8kB serialized, this optimization will mostly hold.

This is not the case when sending larger documents or when sending MirroredConfigSetRequest-s that have multiple content streams. Here's the breakdown:

{noformat}
actualSize=61		byteArraySize=64	waste=3		(4.7% of transmitted bytes)
actualSize=153		byteArraySize=153	waste=0		(0.0% of transmitted bytes)
actualSize=604		byteArraySize=604	waste=0		(0.0% of transmitted bytes)
actualSize=4104		byteArraySize=4104	waste=0		(0.0% of transmitted bytes)
actualSize=6104		byteArraySize=6104	waste=0		(0.0% of transmitted bytes)
actualSize=8104		byteArraySize=8104	waste=0		(0.0% of transmitted bytes)
actualSize=8304		byteArraySize=16464	waste=8160	(49.6% of transmitted bytes)
actualSize=10104	byteArraySize=20064	waste=9960	(49.6% of transmitted bytes)
actualSize=15104	byteArraySize=30064	waste=14960	(49.8% of transmitted bytes)
actualSize=20105	byteArraySize=40066	waste=19961	(49.8% of transmitted bytes)
actualSize=30105	byteArraySize=32768	waste=2663	(8.1% of transmitted bytes)
actualSize=50105	byteArraySize=65536	waste=15431	(23.5% of transmitted bytes)
actualSize=100105	byteArraySize=131072	waste=30967	(23.6% of transmitted bytes)
{noformat}

The {{actualSize}} represents the size of serialized data, {{byteArraySize}} is the size of the returned byte array that contains the serialized data. Up until the boundary of 8192 bytes the optimization works very well, but above this threshold it produces a lot of waste.

I propose to add a conditional in {{ExposedByteArrayOutputStream.byteArray()}} to return the raw buffer only if it's at most 10% larger than the size of serialized data. This will cover all small-doc cases and avoid an array copy for some less egregious cases for larger docs.

Here's the breakdown with the proposed optimization:
{noformat}
actualSize=61		byteArraySize=64	waste=3		(4.7% of transmitted bytes)
actualSize=153		byteArraySize=153	waste=0		(0.0% of transmitted bytes)
actualSize=604		byteArraySize=604	waste=0		(0.0% of transmitted bytes)
actualSize=4104		byteArraySize=4104	waste=0		(0.0% of transmitted bytes)
actualSize=6104		byteArraySize=6104	waste=0		(0.0% of transmitted bytes)
actualSize=8104		byteArraySize=8104	waste=0		(0.0% of transmitted bytes)
actualSize=8304		byteArraySize=8304	waste=0		(0.0% of transmitted bytes)
actualSize=17386	byteArraySize=17386	waste=0		(0.0% of transmitted bytes)
actualSize=24700	byteArraySize=24700	waste=0		(0.0% of transmitted bytes)
actualSize=28550	byteArraySize=28550	waste=0		(0.0% of transmitted bytes)
actualSize=31026	byteArraySize=32768	waste=1742	(5.3% of transmitted bytes)
actualSize=36877	byteArraySize=36877	waste=0		(0.0% of transmitted bytes)
actualSize=43667	byteArraySize=43667	waste=0		(0.0% of transmitted bytes)
actualSize=47067	byteArraySize=47067	waste=0		(0.0% of transmitted bytes)
actualSize=50696	byteArraySize=50696	waste=0		(0.0% of transmitted bytes)
actualSize=57516	byteArraySize=57516	waste=0		(0.0% of transmitted bytes)
actualSize=61110	byteArraySize=65536	waste=4426	(6.8% of transmitted bytes)
actualSize=69145	byteArraySize=69145	waste=0		(0.0% of transmitted bytes)
actualSize=79692	byteArraySize=79692	waste=0		(0.0% of transmitted bytes)
actualSize=89342	byteArraySize=89342	waste=0		(0.0% of transmitted bytes)
actualSize=91734	byteArraySize=91734	waste=0		(0.0% of transmitted bytes)
actualSize=93769	byteArraySize=93769	waste=0		(0.0% of transmitted bytes)

{noformat}


> CrossDC Common: ExposedByteArrayOutputStream optimizes wrong aspect
> -------------------------------------------------------------------
>
>                 Key: SOLR-18415
>                 URL: https://issues.apache.org/jira/browse/SOLR-18415
>             Project: Solr
>          Issue Type: Bug
>          Components: module - crossDC
>    Affects Versions: 10.0, 9.10.1
>            Reporter: Andrzej Bialecki
>            Assignee: Andrzej Bialecki
>            Priority: Major
>             Fix For: main(11.0), 10.1
>
>
> {{MirroredSolrRequestSerializer}} uses a subclass of {{ByteArrayOutputStream}} in an attempt to optimize data transfer by avoiding array copy - instead it passes the whole buffer directly as the output bytes from {{{}serialize(){}}}.
> This sounds like a plausible optimization. However, the initial size of the buffer (32 bytes) is doubled on each overflow as the codec writes to the stream, which means that on average the actual serialized data occupies around half of the buffer, the rest are zeroes. Deserialization will handle these trailing zeros just fine, it's the cost of transmitting roughly twice as much data over the wire that is questionable - this buffer is then passed as-is to the output stream and eventually ends up as network traffic.
> This "optimization" avoids one array copy cost (which reduces GC) but generates multiple costs on the way - not only the empty data has to be sent over the network but also it needs to be stored in source Kafka, in MirrorMaker and in target Kafka, to be finally retrieved by the Consumer (which has to allocate a buffer sufficiently large to fit also the zeroes), only to discard the zeroes during deserialization.
> I propose to get rid of this optimization and use the plain {{ByteArrayOutputStream}} instead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org