[GitHub] [arrow] eerhardt commented on a diff in pull request #13810: ARROW-17330: [C#] Extend ArrowBuffer.BitmapBuilder to improve performance of array concatenation

["eerhardt (via GitHub)" <gi...@apache.org>]

eerhardt commented on code in PR #13810:
URL: https://github.com/apache/arrow/pull/13810#discussion_r1148367033


##########
csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs:
##########
@@ -141,15 +141,17 @@ public TBuilder Append(ReadOnlySpan<T> span)
         {
             int len = ValueBuffer.Length;
             ValueBuffer.Append(span);
-            ValidityBuffer.AppendRange(Enumerable.Repeat(true, ValueBuffer.Length - len));
+            int additionalBitsCount = ValueBuffer.Length - len;
+            ValidityBuffer.Reserve(additionalBitsCount).AppendRange(Enumerable.Repeat(true, additionalBitsCount));

Review Comment:
   If you really wanted to optimize this, we should get rid of the `Enumerable.Repeat` as well.



##########
csharp/src/Apache.Arrow/ArrowBuffer.BitmapBuilder.cs:
##########
@@ -89,6 +89,37 @@ public BitmapBuilder Append(bool value)
                 return this;
             }
 
+            /// <summary>
+            /// Append a span of bits.
+            /// </summary>
+            /// <param name="source">Source of bit span.</param>
+            /// <param name="validBits">Number of valid bits in the bit span</param>
+            /// <returns>Returns the builder (for fluent-style composition).</returns>
+            public BitmapBuilder Append(ReadOnlySpan<byte> source, int validBits)
+            {
+                var length = source.IsEmpty ? validBits : Math.Min(source.Length * 8, validBits);

Review Comment:
   Since this is a public API, I think we should we do parameter validation that ensures `validBits` <= `source.Length * 8`. Or we make it internal.



##########
csharp/src/Apache.Arrow/ArrowBuffer.BitmapBuilder.cs:
##########
@@ -89,6 +89,37 @@ public BitmapBuilder Append(bool value)
                 return this;
             }
 
+            /// <summary>
+            /// Append a span of bits.
+            /// </summary>
+            /// <param name="source">Source of bit span.</param>
+            /// <param name="validBits">Number of valid bits in the bit span</param>
+            /// <returns>Returns the builder (for fluent-style composition).</returns>
+            public BitmapBuilder Append(ReadOnlySpan<byte> source, int validBits)
+            {
+                var length = source.IsEmpty ? validBits : Math.Min(source.Length * 8, validBits);
+
+                // Check if memory copy can be used from the source array (performance optimization for byte-aligned coping)
+                if (!source.IsEmpty && this.Length % 8 == 0)
+                {
+                    EnsureAdditionalCapacity(length * 8);
+                    source.Slice(0, BitUtility.ByteCount(length)).CopyTo(Span.Slice(Length / 8));
+                    
+                    Length += length;
+                    SetBitCount = BitUtility.CountBits(Span, 0, Length);

Review Comment:
   Do we really need to count all the bits again? Can't we just count the bits that were appended and add it to the existing number?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org