[GitHub] [arrow] pitrou commented on a diff in pull request #35825: GH-32723: [C++] Add option to use LARGE* variants of binary types

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #35825:
URL: https://github.com/apache/arrow/pull/35825#discussion_r1210569849


##########
cpp/src/arrow/array/builder_dict.h:
##########
@@ -724,6 +747,7 @@ using BinaryDictionaryBuilder = DictionaryBuilder<BinaryType>;
 using StringDictionaryBuilder = DictionaryBuilder<StringType>;
 using BinaryDictionary32Builder = Dictionary32Builder<BinaryType>;
 using StringDictionary32Builder = Dictionary32Builder<StringType>;
+using BinaryDictionary64Builder = Dictionary64Builder<LargeBinaryType>;

Review Comment:
   @arthurpassos This seems to conflate two different concerns:
   1) being allowed to fit more than 2^31 data bytes in a dictionary (for example more than 2^21 strings of 1kB each)
   2) being allowed to fit more than 2^31 strings in a dictionary
   
   While 1) already seems quite unlikely to me, 2) seems _extremely_ unlikely (shouldn't you just use plain encoding instead?). Do you really have a use case for it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org